As others said you can host Koboldai on your own, and if you don’t have a powerful GPU, anyone can use powerful llm models via the AI horde

man -k

At work today, I just heard the term, “Federated Machine Learning” - where individual corporations keep their proprietary learning data, but make the models shareable. I wonder if something similar could be done with activitypub?

(Feel free to steal this idea if it’s worth stealing)

Like, have you even tried a search or even attempted to look up anything? There is a lot. Most of it boils down to stay under free limits of street paying for compute and run your own. It’s actually very trivial to run a slightly low effective model on a 4090. Liked you can be up and running inside a few minutes if you already know pip

OpenAssistant

petals.dev It’s basically bittorrent for AI. A public swarm for running and training LLMs.

For images there are foss options that you can run locally, like stable diffusion which is so good that it rivals its proprietary counterparts.
But for text it’s a horror, there are some you can try (see gpt4all), but in general chatgpt has no real competition; the foss options are currently very bad, and even the proprietary options from big corporations like bard or llama are pitiful.

but I can’t help but see the fact that there is no alternative to Bing GPT

Regarding this, there are other services that also use GPT for search, here a couple:

https://www.perplexity.ai/

https://www.phind.com/

deleted by creator

what do us poor folks on Linux do?

Run llama.cpp and any of the models listed here, that stuff has been around for months.

TheBloke has a lot of models converted to GGUF format which you need for llama.cpp.

Quick Start Guide (requires Nix, otherwise compile llama.cpp manually):

$ GIT_LFS_SKIP_SMUDGE=1 git clone https://huggingface.co/TheBloke/guanaco-7B-GGUF
$ cd guanaco-7B-GGUF
$ git lfs pull --include=Guanaco-7B.Q4_0.gguf
$ nix run github:ggerganov/llama.cpp -- -m Guanaco-7B.Q4_0.gguf --instruct
> Write haiku about a penguin
 A penguin walks on ice,
 Takes a plunge in the sea,
 Hides his feet from me!

I installed Edge on Linux because it’s fun watching Linux people get angry about it

There are quite a few of them out there. Just a few from my notes (in reverse chronological order)

• https://github.com/persimmon-ai-labs/adept-inference
• https://github.com/shellc/laozy
• https://github.com/imartinez/privateGPT
• https://github.com/Josh-XT/AGiXT
• https://github.com/tloen/alpaca-lora

The self-hosted AI/ML system has been here for a while. Granted, the vast majority of them require downloading pre-trained models before they can be used due to how much it costs to build a system of weights from scratch.

I sometimes wonder if it would be possible to build and train a truly open source model with BOINC or something. The last 30 years of history show that it’s entirely feasible to build a massively distributed computing cluster, why not leverage this to build a model? I know how naive that sounds immediately after writing it, mostly because of the difficulty of getting a large enough training data set, which unfortunately has risk written all over it (read: people poisoning the model, ala Microsoft’s experiment with Tay on birbsite some years back).

i don’t know a single thing about whatever these things your talking about are, but i just want to say, you’re the first person i’ve ever seen suggest that edge isn’t a joke.

Chatbot arena ranks LLM, including some truly open source ones (apache license)

Regarding UI, maybe look here?

deleted by creator

I think it’s far more likely that easier solutions to hosting your own LLM/ChatGPT/etc will be pushed, rather than someone (or a group of someones) offering and maintaining it - as you mentioned, that takes a lot of resources. Think about the issues that various Fediverse software has had in terms of keeping up with the load sometimes, and LLMs use way more than most Fediverse applications. Especially if you want it to be as snappy as the current ones already out there.