- replace json (storing data as strings? really?) with a binary format like protobuf, or better yet parquet
- replace redis with duckdb for zero-copy reads
- replace pandas with polars for faster transformations
- use asynchronous, modern web framework for microservices like fastAPI
- Tune xgboost CPU resource usage with semaphores