This custom_node for ComfyUI adds one-click "Virtual VRAM" for any UNet and CLIP loader as well MultiGPU integration in WanVideoWrapper, managing the offload/Block Swap of layers to DRAM *or* VRAM to maximize the latent space of your card. Also includes nodes for directly loading entire components (UNet, CLIP, VAE) onto the device you choose

Privacy-first AI ecosystem for Android. Run GGUF models offline or access 100+ cloud models via OpenRouter. Features 11 premium offline voices, extensible plugins, and dynamic DataHub for context injection. No subscriptions, no data harvesting—just AI on your terms.

A quant trading system platform based on FinGPT, demonstrating new applications of large pre-trained Language Models in quantitative finance.

Experimental interface environment for open source LLM, designed to democratize the use of AI. Powered by llama-cpp, llama-cpp-python and Gradio.

Discover the Best AI Models for Your PC

Run Llama 3.3, DeepSeek-R1, Phi-4, Gemma 3, Mistral Small 3.1, and other state-of-the-art language models locally with scorching-fast performance. Inferno provides an intuitive CLI and an OpenAI/Ollama-compatible API, putting the inferno of AI innovation directly in your hands.

Create out symbolic links for the GGUF Models in Ollama Blobs. for use in other applications such as Llama.cpp/Jan/LMStudio etc. / 将 Ollama GGUF 模型文件软链接出，以便其他应用使用。

Simple Python Script that converts the Weight of a GGUF Model to a PyTorch Tensor

Convert and quantize llm models

🤖 Serverless AI on Hedera! Smart contracts run LLMs ⚙️ Distilled GGUF on IPFS. Pay crypto 🪙 → get replies 💬. 99.9% cheaper 💸 than big-tech APIs. Decentralized ⛓️ no censorship. “Gov can’t stop it. Nobody can.” 🧠 – Sir Charles Spikes , Cincinnati,Ohio| TELEGRAM: @SirGODSATANAGI✉️ SirCharlesspikes5@gmail.com #AGI #NoPC

Auto GGUF Converter for HuggingFace Hub Models with Multiple Quantizations (GGUF Format)

This is a Bash script to automatically launch llama-server, detects available .gguf models, and selects GPU layers based on your free VRAM.

A high-performance local service engine for large language models, supporting various open-source models and providing OpenAI-compatible API interfaces with streaming output support.

Automate bulk downloads of Hugging Face LLMs with retry logic, manifest export, checksum validation, and usage reporting. Ideal for managing GGUF models at scale.

A web UI for managing multiple models with llama-server.exe on windows

A modular CLI chatbot with Discord webhook integration optimized for edge devices using GGUF quantized models.

A fine-tuned legal QA system using Zephyr or LLaMA with GGUF support. Real-time inference via Hugging Face API or local model hosting.

GGUF model list & tools

This repository demonstrates how to use outlines and llama-cpp-python for structured JSON generation with streaming output, integrating llama.cpp for local model inference and outlines for schema-based text generation.

A fast and reliable model downloader written in Go for downloading HuggingFace models.