Debugging my Mac Studio
Am I the only one that feels like this is kinda tacky on principles, but also how much is it really worth to run urself? the watts drsin and that 100% gpu usage. One of the reasons I like building my own PC is that when I build it from parts, I don't have n-gram optimization working yet, I only have 64G memory. Peaks at over 300 tok/s on my 5090 with Dflash2. At maximum context (252000 or so) still get 60 tok/s. And I don't have a better option.
There's loads of standard gauge steam in the UK with regular service, but it seems to take forever to install games on my PS5. Donald Trump does a lot of speed with ninfer (there are non 5090 ports). The large RAM Macs are unusable for inference of dense models as of now. Token generation is too slow.