Would you like to run 235B parameter LLMs at home, but your lowly $10,000 budget restricts you to “consumer GPUs that can barely handle 70B parameter models”? This was the situation developer David Noel Ng found himself in, until he stumbled across an “obviously fake” Nvidia Grace-Hopper platform being sold on Reddit, of all places. Ng took the gamble and, according to his blog post, it paid off royally. He’s managed, with a bit of tinkering and fixing up, to get an enterprise system that would usually cost ~ $80,000 for a tenth of that sum. The included 960GB of LPDDR5X memory, alone, is now worth more than he paid for the full system. Hilariously, he even lowballed the seller, noting the original listing on Reddit was for 10,000 EUR before offering just 7,000 EUR.
