FHIR Group Member Incremental Addition with ClickHouse

System Design · sequence diagram · MIT

Illustrates the PATCH flow for incrementally adding members to a FHIR Group resource, leveraging ClickHouse for event storage and MongoDB for metadata upda

Source: https://github.com/icanbwell/fhir-server/blob/97ba98f5a1285179af54cbd4a17fb9334a728d0c/readme/clickhouse.md
Curated by icanbwell
FHIR PATCH ClickHouse MongoDB Sequence Diagram System Design Event Sourcing

Mermaid source

sequenceDiagram
    participant Client
    participant API as FHIR API
    participant PatchOp as PATCH Handler
    participant Handler as ClickHouseGroupHandler
    participant Repo as GroupMemberRepository
    participant CH as ClickHouse
    participant Context as httpContext
    participant Mongo as MongoDB

    Client->>API: PATCH /4_0_0/Group/{id}<br/>[{op: "add", path: "/member/-", value: {...}}]
    API->>PatchOp: handle PATCH

    Note over PatchOp: Detect member operations
    PatchOp->>PatchOp: Parse member ops<br/>eventsToAdd[]<br/>eventsToRemove[]

    PatchOp->>Handler: writeEventsAsync(groupId, added, removed)
    Handler->>Repo: appendEvents(events)
    Repo->>CH: INSERT INTO Group_4_0_0_MemberEvents
    CH->>CH: Materialized view updates
    CH-->>Repo: Success
    Repo-->>Handler: Success
    Handler-->>PatchOp: Success

    PatchOp->>Context: Set GROUP_MEMBER_EVENTS_WRITTEN flag
    PatchOp->>Mongo: Update metadata only<br/>(versionId, lastUpdated)

    Note over PatchOp: PostSave handler sees flag,<br/>skips member processing

    PatchOp-->>API: Success
    API-->>Client: 200 OK

What this diagram shows

This diagram illustrates the sequence of operations for incrementally adding members to a FHIR Group resource using a PATCH request. It details how a FHIR API, a dedicated PATCH handler, a ClickHouse-based event handler, and MongoDB interact to process member additions efficiently, focusing on event-sourcing principles for member data and separate metadata updates.

When to use it

Use this pattern when implementing a FHIR server that requires efficient, high-volume updates to resource collections (like group members) and needs to decouple member data storage from the primary resource document, potentially for performance or analytical purposes. It's suitable for systems adopting an event-sourcing approach for specific resource attributes.

How to adapt it for your project

This flow can be adapted by changing the underlying event store (e.g., Kafka, other databases), modifying the PatchOp logic for different FHIR resource types or operations, or integrating additional validation steps. The GROUP_MEMBER_EVENTS_WRITTEN flag mechanism can be generalized for other partial updates to avoid redundant processing in post-save hooks.

Key concepts