An idea for saving cost. Local AI model use.
So perhaps I could be fooling myself, or maybe this is a bad idea...
But why not allow us to use local models as the helper? If we just added a pipeline to a local model as the brain of the helper, and kept all the resources that the old helper uses as a reference... That might alleviate a lot of stress and some cost off the function of the AI itself, which could save some money would it not?
I'm not super savvy on all the technical background required but as a concept I think this might at least somewhat help save on the COST side of things.
Thoughts?
2 replies
I would guess like many things, the perchance dev(s) have too much on the plate already and can only deal with the most pressing issues. That idea sounds good - a bridge for vscode or something that would allow outside programming assistance (which is probably somewhat viable through copy and paste now, though it would be clunky). The issue is just that though, I suggest vscode because that is what I use and you can implement all the popular AI assistants in vscode - but that is just one of many ways you could implement outside resources, and providing those code mods and plugins is a project in it's own.
I assume perchance is avoiding a donation system, but the reality is it may be a best case - have the funds needed to be able to prioritize projects by another factor than needing to trim expenses, which are unlikely to go down (not much ever does).
That's fair, but a big thing I'm also thinking of is server load. I know many people run several page tabs at once (A person I was talking to was running 10 helpers at once on the same project... Don't ask. He was doing a lot of stuff at once.) and that rubbed me a bit bad since perchance is already struggling with high server usage.
Being able to use local models (or the like) would really help remove a lot of server load. (As well as cost... The cost was more the after-thought during the conception of the idea, but I figured people always think money-first.)