No posts match your filters.
Split the Q-head into a state value and an advantage, then subtract the mean advantage so the two streams stay identifiable - the network stops having to learn the value of every action in every state (Wang et al., 2016). Read more
Monte Carlo policy gradient - collect full episodes, compute returns, and reinforce actions that led to high reward. The original policy gradient method (Williams, 1992). Read more
Notes on optimising the policy directly - gradient ascent on J(θ), why the environment’s transition probability falls out of the gradient, and the policy gradient theorem derived two ways. Read more
Same loop as DQN, one change in the target - the online net picks the action and the target net scores it, so the max stops flattering itself (van Hasselt et al., 2016). Read more
A plain-English glossary for the vocabulary the rest of NeatRL uses - agent, environment, observation, state, action, reward, return, policy and value. Read more
Tabular off-policy TD control - a Q-table for small discrete Gym environments like FrozenLake and CliffWalking (Watkins, 1989). The foundation DQN later approximates with a network. Read more
Deep Q-Network - a neural network approximates Q(s, a), stabilised with experience replay and a frozen target network (Mnih et al., 2015). Read more
A public local LLM benchmark / LLM benchmark leaderboard for edge boards: a reproducible systems bench for small open-weight models, measured under locked power modes. Read more
6 small MoE/hybrid LLMs benchmarked on a Mac Mini M4 16GB across a full ctx=256 → 30,720 sweep: llama.cpp Metal vs MLX-LM. The ranking is context-dependent — SmallThinker-4B-A0.6B hits 130 tok/s at short context but retains only 43.5 %, while LFM2.5-8B-A1B overtakes it at the long-context end. Read more
A practical walkthrough of mDNS, Zeroconf, and Apple’s AWDL: how AirDrop finds your phone, how smoltorrent discovers Raspberry Pis with no hardcoded IPs, and how a Swift sidecar bridges Python to Apple’s peer-to-peer WiFi stack. Read more
5 Bonsai-family 1–1.58bit LLMs benchmarked across 4 power modes on Jetson Orin Nano Super 8GB. 25W sweet spot: 47–48% more tok/s than 15W, best output tok/J for all sub-4B models. Read more
Build a 3-node Jetson Orin Nano Super 8GB cluster with active cooling. Real numbers: ~759 Mbps per link (gigabit), peak 58.3°C across all 3 nodes under full 18-core sustained load, zero throttling at 1728 MHz throughout. Read more
8 tiny LLMs benchmarked across 4 power modes on Jetson Orin Nano Super 8GB: llama.cpp vs Ollama. 25W sweet spot: 43% more tok/s than 15W, better tok/J than MAXN. Read more
An ablation study of GRPO reward signals for 64-token Reddit TL;DR summarization across Qwen2.5-0.5B and LFM-2.5-350M on Apple Silicon. Read more
A 942 MB checkpoint. Four Raspberry Pis. ~1.5 min gather. No single point of failure. A deep dive into smoltorrent - a distributed checkpoint sharding system built over raw TCP with replication, SHA-256 integrity verification, mDNS discovery, and Prometheus monitoring. Read more
Build a 4-node Raspberry Pi 4B cluster with UCTRONICS enclosure, PoE+ hats, and TP-Link LS110P PoE switch. Real numbers: 94.4 Mbps per link (100 Mbps switch ceiling), 62.3°C under full 16-core load, zero throttling at 1800 MHz throughout. Read more
Wire Mac minis into a high-bandwidth local Thunderbolt cluster for distributed training and inference with zero cloud egress cost, low latency, and direct control over cluster networking. Read more
Hugging Face