Data Flow in a Visualization Tool

Data Pipelines · flowchart diagram · BSD-3-Clause

Illustrates the data flow from various sources through a GraphQL layer, configurator, and renderer, with persistence for chart configurations.

Source: https://github.com/visualize-admin/visualization-tool/blob/bfde2d2a338c3a3517f8dce502ea147fc00b4ebc/readme/domain.md
Curated by visualize-admin
data flow visualization graphql sparql d3.js frontend backend

Mermaid source

flowchart TD
    sources["Data Sources<br>(RDF/SPARQL endpoints)"]
    graphql["GraphQL Layer<br>(resolvers, data transforms)"]
    configurator["Chart Configurator<br>(dataset selection and config UI)"]
    renderer["Chart Rendering<br>(D3.js)"]
    persistence["Persistence<br>(Prisma/PostgreSQL)"]

    graphql -->|SPARQL queries| sources
    sources -->|RDF results| graphql

    configurator -->|metadata queries| graphql
    graphql -->|cube metadata| configurator

    renderer -->|observation queries| graphql
    graphql -->|observation data| renderer

    configurator -->|live config preview| renderer

    configurator -->|save chart config| persistence
    persistence -->|load chart config| configurator

    persistence -->|load chart config| renderer

What this diagram shows

This diagram illustrates the end-to-end data flow within a visualization application. It details how data is fetched from RDF/SPARQL sources, processed by a GraphQL layer for transformations and metadata, configured via a UI, rendered using D3.js, and how chart configurations are persisted.

When to use it

Use this diagram to understand or design data pipelines for visualization tools, especially those involving semantic web data (RDF/SPARQL), GraphQL APIs, and client-side rendering. It's suitable for outlining system architecture, data transformation steps, and user interaction flows for chart creation.

How to adapt it for your project

Adapt this diagram by swapping data sources (e.g., REST APIs, databases), replacing GraphQL with other API layers, or changing the rendering library. You can add more steps for data validation, caching, or real-time updates. Extend the persistence layer to include user authentication or versioning for configurations.

Key concepts