I’ve bwem waning to try it out considering it has unified memory. What model are you using and what are you running with? I would be thinking something like a small qwen on llama.cpp
I have an OLED, so, slightly better specs in a few fairly minor ways than an LED.
I am using Bazzite, managing the LLMs inside of Alpaca, which is a flatpak, so it works easily with Bazzite’s ‘containerize everything’ approach.
And uh yep, I’m running Qwen3, the… I think its 8B param variant.
I actually told it its HW and SW environment, told it to generate a context prompt so it just always knows that, then asked it to optimize its own settings… and it did come up with settings that make it run a either a bit generally better, or in altetnate sort of modes… I just made a variant ‘Focused’ and variant ‘Contemplative’, first one for mechanistic, step 1 2 3 type thinking, secons one for larger conceptualization questions.
Though I think I need to tweak the contemplative variant to be a biiiit less imaginative, it tends to hallucinate and conrtadict itself a bit too much.
I’ve also been able to like, tell it to read an updated websitr with more modern syntax for GDScript, and tell ot to make itself a context prompt that tells it about it, and then it roughly just ‘knows’ that… I think the training data is 1 to 2 years out of date now, so occasional little patchwork fixes like that seem to work?
I’ve bwem waning to try it out considering it has unified memory. What model are you using and what are you running with? I would be thinking something like a small qwen on llama.cpp
I have an OLED, so, slightly better specs in a few fairly minor ways than an LED.
I am using Bazzite, managing the LLMs inside of Alpaca, which is a flatpak, so it works easily with Bazzite’s ‘containerize everything’ approach.
And uh yep, I’m running Qwen3, the… I think its 8B param variant.
I actually told it its HW and SW environment, told it to generate a context prompt so it just always knows that, then asked it to optimize its own settings… and it did come up with settings that make it run a either a bit generally better, or in altetnate sort of modes… I just made a variant ‘Focused’ and variant ‘Contemplative’, first one for mechanistic, step 1 2 3 type thinking, secons one for larger conceptualization questions.
Though I think I need to tweak the contemplative variant to be a biiiit less imaginative, it tends to hallucinate and conrtadict itself a bit too much.
I’ve also been able to like, tell it to read an updated websitr with more modern syntax for GDScript, and tell ot to make itself a context prompt that tells it about it, and then it roughly just ‘knows’ that… I think the training data is 1 to 2 years out of date now, so occasional little patchwork fixes like that seem to work?