AI Topic: Local Inference

Keras is an open-source, high-level deep learning API that enables building, training, and deploying neural networks across JAX, TensorFlow, and PyTorch backends.

An open-source, high-performance library for serving and running large language models with GPU-optimized inference and efficient memory and batch management.

Self-hosted open-source AI API server that exposes unified REST endpoints and supports multiple LLM providers for integration into applications.

Open-source AI-powered browser that automates web tasks via natural language agents while prioritizing privacy and local model support.

End-to-end, open-source recipe to train and serve a small chat LLM (~560M params) for about $100 on one 8×H100 node, with tokenizer, pretrain→midtrain→SFT→optional RL, FastAPI web UI, and a KV-cached inference engine.

Osaurus is a local-first AI runtime optimized for Apple Silicon that runs open-source models on Mac with privacy and no cloud dependency.

Run local LLMs, chat with documents, and power apps using a local AI server.