API Endpoint Flow for Distributed Feature Engineering and Inference

ML & AI · flowchart diagram · unknown license

Illustrates the API endpoint flow for a system handling distributed feature engineering and inference, including health checks, prediction, and feature ret

Source: https://github.com/jayeshakhare33/DiFE/blob/9e22a00e5e9a1360caa835b0e1020e718cac9828/readme/architecture/PROJECT_DIAGRAMS.md
Curated by jayeshakhare33
FastAPI Machine Learning Distributed Systems Feature Engineering Model Inference Redis PostgreSQL

Mermaid source

flowchart TD
    Client[Client Application] -->|HTTP Request| API[FastAPI Server<br/>:8000]
    
    API --> Route{Route Handler}
    
    Route -->|GET /health| Health[Health Check<br/>Return Status]
    Route -->|POST /predict| Predict[Predict Endpoint]
    Route -->|GET /features| Features[Get Features Endpoint]
    
    Predict --> ValidateReq{Validate<br/>Request Body}
    ValidateReq -->|Invalid| Error1[400 Bad Request]
    ValidateReq -->|Valid| CheckFeat{Features<br/>Available?}
    
    CheckFeat -->|No| Route1[Route 1: Distributed<br/>Feature Engineering]
    CheckFeat -->|Yes| Route2[Route 2: Distributed<br/>Inference]
    
    subgraph "Route 1: Distributed Feature Engineering"
        Route1 --> DistFE[Distribute Nodes<br/>to Feature Workers]
        DistFE --> FE1[Feature Worker 1<br/>Calculate Features]
        DistFE --> FE2[Feature Worker 2<br/>Calculate Features]
        DistFE --> FE3[Feature Worker N<br/>Calculate Features]
        
        FE1 --> Access1[Access Central DB<br/>PostgreSQL & Neo4j]
        FE2 --> Access2[Access Central DB<br/>PostgreSQL & Neo4j]
        FE3 --> Access3[Access Central DB<br/>PostgreSQL & Neo4j]
        
        Access1 --> Store1[Store Features<br/>in Redis]
        Access2 --> Store2[Store Features<br/>in Redis]
        Access3 --> Store3[Store Features<br/>in Redis]
        
        Store1 --> Route2
        Store2 --> Route2
        Store3 --> Route2
    end
    
    subgraph "Route 2: Distributed Inference"
        Route2 --> BatchSplit[Split Transaction Batch<br/>e.g., 1000 txns → 5×200]
        BatchSplit --> Inf1[Inference Worker 1<br/>200 transactions]
        BatchSplit --> Inf2[Inference Worker 2<br/>200 transactions]
        BatchSplit --> Inf3[Inference Worker 3<br/>200 transactions]
        BatchSplit --> Inf4[Inference Worker 4<br/>200 transactions]
        BatchSplit --> Inf5[Inference Worker 5<br/>200 transactions]
        
        Inf1 --> GetFeat1[Get Features<br/>from Redis]
        Inf2 --> GetFeat2[Get Features<br/>from Redis]
        Inf3 --> GetFeat3[Get Features<br/>from Redis]
        Inf4 --> GetFeat4[Get Features<br/>from Redis]
        Inf5 --> GetFeat5[Get Features<br/>from Redis]
        
        GetFeat1 --> CreateTuple1[Create Input Tuples<br/>for GNN Model]
        GetFeat2 --> CreateTuple2[Create Input Tuples<br/>for GNN Model]
        GetFeat3 --> CreateTuple3[Create Input Tuples<br/>for GNN Model]
        GetFeat4 --> CreateTuple4[Create Input Tuples<br/>for GNN Model]
        GetFeat5 --> CreateTuple5[Create Input Tuples<br/>for GNN Model]
        
        CreateTuple1 --> RunInf1[Run Inference<br/>GNN Model]
        CreateTuple2 --> RunInf2[Run Inference<br/>GNN Model]
        CreateTuple3 --> RunInf3[Run Inference<br/>GNN Model]
        CreateTuple4 --> RunInf4[Run Inference<br/>GNN Model]
        CreateTuple5 --> RunInf5[Run Inference<br/>GNN Model]
        
        RunInf1 --> Aggregate[Aggregate Results<br/>from All Workers]
        RunInf2 --> Aggregate
        RunInf3 --> Aggregate
        RunInf4 --> Aggregate
        RunInf5 --> Aggregate
    end
    
    Aggregate --> Format[Format Response]
    Format --> Return1[200 OK<br/>Fraud Scores]
    
    Features --> ValidateQuery{Validate<br/>Query Params}
    ValidateQuery -->|Invalid| Error2[400 Bad Request]
    ValidateQuery -->|Valid| QueryFeat[Query Features<br/>from Redis]
    QueryFeat --> Return2[200 OK<br/>Feature Data]
    
    Health --> Return3[200 OK<br/>Service Status]
    
    Return1 --> Client
    Return2 --> Client
    Return3 --> Client
    Error1 --> Client
    Error2 --> Client
    
    style Client fill:#e1ffe1
    style API fill:#fff5e1
    style Route1 fill:#fff5e1
    style Route2 fill:#e1f5ff
    style Return1 fill:#e1f5ff
    style Return2 fill:#e1f5ff
    style Return3 fill:#e1f5ff
    style Error1 fill:#ffe1e1
    style Error2 fill:#ffe1e1

What this diagram shows

This diagram shows the complete API endpoint flow for a system that performs distributed feature engineering and distributed inference using a FastAPI server. It details how client requests for health checks, predictions, and features are handled, including request validation, feature availability checks, and the orchestration of feature workers and inference workers accessing databases like PostgreSQL, Neo4j, and Redis, and running GNN models.

When to use it

Use this diagram when designing or documenting an API-driven machine learning system that requires scalable, distributed processing for feature engineering and model inference. It's particularly useful for systems dealing with large datasets, complex feature calculations, or real-time predictions with Graph Neural Networks.

How to adapt it for your project

This diagram can be adapted by changing the specific ML model (e.g., from GNN to CNN/RNN), modifying the data sources (e.g., adding Kafka, S3), or adjusting the worker distribution strategy. The validation and error handling steps are generic and can be applied to any API. The 'Features Available?' check can be extended to include caching strategies or feature store lookups.

Key concepts