Local inference
How far a 12 GB card can be pushed: quantisation levels, context length, offload strategy, and where quality falls off a cliff rather than degrading gently. The output is usually a table and a strong opinion.
The lab
A workbench, a GPU that is old enough to be interesting, and a cloud tenant nobody will miss if I break it.
Gribbite exists because reading about a system is not the same as running one. Most of what gets published about AI infrastructure and AI security is either a vendor demo or a benchmark on hardware nobody owns. This lab is the opposite: constrained hardware, real constraints, and results that include the parts that did not work.
Everything here is non-production and deliberately disposable. That is the point — it means an experiment can be destructive.
The rig
Modest on purpose. If something only works on a card that costs more than a car, it is not a useful finding for most people.
- gpu
- NVIDIA Titan X (Maxwell) — 12 GB VRAM. Old enough that nothing is optimised for it, which makes it an honest floor for local inference.
- system
- 32 GB RAM, NVMe storage for model weights, Windows with WSL for the Linux-only pieces.
- inference
- llama.cpp with GGUF quantisations. Qwen-family models are the current baseline; others get swapped in for comparison.
- cloud
- A dedicated Microsoft 365 and Azure lab tenant with its own domain, hybrid identity, and no connection to anything that matters.
- tooling
- PowerShell for automation, KQL for anything involving telemetry, and Markdown for everything else.
Areas of experimentation
These overlap constantly. An agent experiment turns into an identity problem; a detection experiment turns into a data-quality problem.
Generative AI in practice
Prompt structure, retrieval, evaluation. Less interest in what a model can do once, more in what it does reliably across a hundred runs with messy input.
Agents and agentic workflows
Narrow agents with real credentials and a small tool surface. Scoping, failure handling, and what happens at the boundary where the model decides to call something.
AI security
Prompt injection through retrieved content, tool-call abuse, and exfiltration paths that open up the moment a model is given context it did not fetch itself. Agents are workloads, and workloads get threat models.
Detection engineering
Writing detections in KQL, running simulated activity against them, and paying attention to the false-positive rate before a rule ships anywhere.
Identity and automation
Entra ID governance, access packages, lifecycle workflows, and the PowerShell and Graph automation that holds it together — including what breaks when a directory gets messy.
How work happens here
Small enough to finish
An experiment that takes a weekend gets done. One that takes a quarter becomes a folder of half-finished notes. Scope is set accordingly.
Reproducible or it did not happen
Every project page carries the hardware, the versions, and the commands. If you cannot repeat it on your own machine, it is an anecdote.
The failures are the useful part
The configurations that fell over, the quantisation that produced fluent nonsense, the detection that fired on every login — those get written down with the same care as the results that worked.
Nothing sensitive, ever
No employer material, no customer data, no production telemetry. Everything on this site comes from equipment and tenants that belong to the lab.
Start with the projects
Each one carries its setup, what was measured, and what turned out to be wrong.