Skip to content
@csorchard

Concurrent Systems Engineering

single node @dborchard → multi-node @dsorchard → multi-core @csorchard
README.md

🌱 The Foundation

Action Precedes Motivation

Read More

Core Data Structure

  • Regular Data structure
    • Randomized: Quick Sort, Treap, SkipList
    • Misc: List, Stack, Queue, Priority Queue (taken from gods)
    • Sorted: Binary Search Tree
  • Systems Data structure
    • Hash-Based: Probing vs Chaining in Hashmap
      • Approximate: Bloom Filter, Count Min Sketch, HyperLogLog
      • Rebalancing Map: Consistent Hashing, ChordDHT, Range Based Partitioning, Extendable Hashing
      • Hash Functions: Cuckoo Hash, Murmur Hash,
    • Sorted: BTree, Binary Search Tree, Treap, Red Black Tree, B Epsilon Tree
    • Multi Dimension: KD Tree, Z-Index, Hilbert Curve
    • Sampling: Reservoir Sampling
    • Stream: Sliding window
    • Difference: Merkal Tree
    • Time: Hashed Wheel Timer
  • Concurrent Data Structures:
    • Skip List: Regular, Lazy, LockFree, Arena Skiplist
    • BTree
    • Radix Tree: ART, VART, Trie
    • Fast Succinct Trie: SuRF
  • External Memory Algorithms
    • External Hashing
    • External Sorting

Performance engineering tools

  • Performance Analysis
    • Heap Dump
    • Trace
    • Jmeter, YCSB
    • VRPC (vector clock)
    • Heap View
    • Write Benchmark
    • Lotsaa
    • GC frequency, threshold
    • Git Bisect
    • Explain Analyze
    • OpenTelemetry Trace

Advanced Data Structures

  • Arena based skip list
  • Roaring BitMap
  • Zone Map

🌿 The Plant

Read, Extract components, and Learn the inner workings.

Read More

From MatrixOrigin

  • MatrixOrigin Systems Data Structures Usage
    • HyperLogLog: Used to find NDV before writing segment meta header (MO Write SST with Metadata including NDV)
    • Bloom Filter: Used for fast range scan through blocks (MO Runtime Filter in Storage Engine)
  • MatrixOrigin Join
    • Hash Map
    • Memory Pool
    • Reservoir Sampling

Misc

  • Storage Engine: WAL, Btree, CoW Btree, LSM Tree
  • Memtable using Skip List

🌳 The Tree

Pick a favorite storage engine/main memory system and shrink.

Read More

Storage Engines

  • Dgraph ristretto
  • Dgraph badger: WiscKey, Txn, Memory allocation part in pkg z.
  • BigCache:

💧 The Resources

Revisit papers/slides. Build counterexamples. Think like a scientist.

Read More

Reading

Read

👨‍🌾 Cultivating Knowledge

Write what you understood. (Maybe later)

Read More

Teachings

  • Paper Reading
    • A method for implementing lock-free shared data structures
    • WiscKey: Separating Keys from Values in SSD-conscious Storage
    • A Critique of Snapshot Isolation

🥭 The Fruit

Mark your achievements.

Read More

Core Implementations

Pinned Loading

  1. sds_use sds_use Public

    Systems Data Structure Usages in Real world projects

    Go

  2. sds sds Public

    Data structures and algorithms I came across in database/systems programming.

    Go

  3. perf_analysis perf_analysis Public

    Performance analysis of Golang Project

  4. skiplist_impls skiplist_impls Public

    Learning Regular, Lazy, Lock Free, Arena skiplist in Golang

    Go 2

Repositories

Showing 10 of 30 repositories
  • skiplist_impls Public

    Learning Regular, Lazy, Lock Free, Arena skiplist in Golang

    csorchard/skiplist_impls’s past year of commit activity
    Go 2 0 0 0 Updated Jun 18, 2024
  • .github Public
    csorchard/.github’s past year of commit activity
    0 0 0 0 Updated Jun 13, 2024
  • perf_analysis Public

    Performance analysis of Golang Project

    csorchard/perf_analysis’s past year of commit activity
    0 0 0 0 Updated May 30, 2024
  • mm-go Public Forked from joetifa2003/mm-go

    Generic manual memory management for golang

    csorchard/mm-go’s past year of commit activity
    Go 0 MIT 5 0 0 Updated Apr 4, 2024
  • sds Public

    Data structures and algorithms I came across in database/systems programming.

    csorchard/sds’s past year of commit activity
    Go 0 0 0 0 Updated Mar 31, 2024
  • sds_use Public

    Systems Data Structure Usages in Real world projects

    csorchard/sds_use’s past year of commit activity
    Go 0 0 0 0 Updated Mar 27, 2024
  • awesome-go Public Forked from avelino/awesome-go

    A curated list of awesome Go frameworks, libraries and software

    csorchard/awesome-go’s past year of commit activity
    Go 0 MIT 12,269 0 0 Updated Mar 18, 2024
  • go-profiler-notes Public Forked from DataDog/go-profiler-notes

    felixge's notes on the various go profiling methods that are available.

    csorchard/go-profiler-notes’s past year of commit activity
    Jupyter Notebook 0 CC-BY-SA-4.0 212 0 0 Updated Mar 16, 2024
  • zallocator Public Forked from al8n/zallocator

    Amortizes the cost of small allocations by allocating memory in bigger chunks.

    csorchard/zallocator’s past year of commit activity
    Rust 0 Apache-2.0 1 0 0 Updated Mar 6, 2024
  • stretto Public Forked from al8n/stretto

    Stretto is a Rust implementation for Dgraph's ristretto (https://github.com/dgraph-io/ristretto). A high performance memory-bound Rust cache.

    csorchard/stretto’s past year of commit activity
    Rust 0 Apache-2.0 28 0 0 Updated Feb 12, 2024
View all repositories

Top languages

Loading…

Most used topics

Loading…