*(In this case, we don't think about replication, but in real life, we can use both at the same time)

  1. Ma'lumotlarni bitta joyda saqlash juda xavfli va DB sekinlashishiga olib keladi. Shuningdek, serverni vertical scale qilish muammosi kelib chiqadi agar ma'lumotlarimiz TB'lab bo'lsa.

<aside> 💡

Buni oldini olishning yo'llaridan biri "partition"larga bo'lish. Masalan: hash(key) mod N : P0: 1 - 100k P1: 100k - 200k P2: 200k - 300k ... Xuddi shunday partitionlarga bo'lib tashlasak bo'ladi. Shu usulga Document-Partitioned Index (a.k.a. Local Index) deyiladi.

</aside>

Used in: elasticsearch

Scale qilish oson, write query faqatgina bitta shardga ta'sir qiladi. Lekin, gender ="male" bo'lgan userlarni olmoqchi bo'lsam, hamma partition'ga bir boshidan request yuborib chiqishimga to'g'ri keladi. Bu ishni ozgina sekinlashtiradi.

  1. Buni fix qilish uchun, partitionni index bo'yicha emas "term / keyword" bo'yicha bo'lsak bo'ladi. Masalan:

    P0: barcha "male" userlar P1: barcha "female" userlar P2: barcha "others" userslar

Ushbu usulga Global Index (Term-Partitioned Index) deyiladi.

Used in: HBase, Cassandra

Partition Rebalancing

is the process of redistributing data across nodes in a distributed database or messaging system to ensure a balanced load, optimal performance, and fault tolerance.

Methods:

1. Fixed Partitions

We choose the number of partitions in advance (e.g., 10, 16, 64), and every piece of data is assigned to one of those partitions permanently. But we can assign partitions to a node dynamically. This is how we do it:

1. Count partitions (fixed number).
2. Count nodes.
3. Compute ideal = partitions / nodes.
4. Move partitions until each node has ~ideal.

2. Dynamic Partitions

Partitions (shards) are not fixed. They split and merge automatically as data grows or shrinks.


Fixed partitioning has two big problems:
Hotspots (one partition getting all the traffic)
No automatic rebalancing
Dynamic partitioning fixes both.