I say let’s keep up this momentum and continue making this a space people want to be in and engage with.

We’re already off to a strong start, let’s commit and see what new corner of the internet we can define for ourselves.

For anyone unaware, this is probably one of the better short and sweet explanations in regards to what HuggingFace is.

It is a hub for many code repositories hosting AI specific files and configurations, which has become a core ecosystem of many artificial intelligence breakthroughs, platforms, and applications.

Hey! Appreciate your post. The AI Horde has been one of my favorite projects to see evolve over the course of this year. Consider me subbed.

For myself (and others not as knowledgeable on the project), do you think you could briefly describe the main differences between how The AI Horde approaches crowd compute / inference compared to something like Petals? I know you mentioned here that the horde doesn't do training. Is that the biggest difference to note?

Thanks again for your contribution to democratizing AI. Excited to see what The AI Horde can do with more supporters. I'll be dedicating a few more nodes when I have a chance to spin them up.

This will be a fine-tuned model, so it may inherit some of the permissions and license agreements as its foundation model and have other implications depending on your country or local law.

You are correct, if we chose Llama 2 - the fine-tune derivative may be subject to their original license terms. However, Apache 2.0 would apply and transfer to something like a fine-tuned version of Mistral, since its base license is also Apache 2.0.

If there is enough support - I'd be more than open to creating an entirely new foundation model family. This would be a larger undertaking than this initial fine-tuning deployment, but building a completely free FOSAI foundation family of models was the penultimate goal of this project so if this garners enough attention I could absolutely put energy and focus into creating another Mistral-like product instead of splashing around with fine-tuning.

Whatever would help everyone the most! I like where you're thinking though, I'm going to update the thread to include an option to vote for a new foundation family instead. At the end of the day, it's likely I'll do all of the above - I'm just not sure in what order yet..

F

Mistral seems to be the popular choice. I think it's the most open-source friendly out of the bunch. I will keep function calling in mind as I design some of our models! Thanks for bringing that up.

I am actively exploring this question.

So far - it’s been the best performing 7B model I’ve been able to get my hands on. Anyone running consumer hardware could get a GGUF version running on almost any dedicated GPU/CPU combo.

I am a firm believer there is more performance and better quality of responses to be found in smaller parameter models. Not too mention interesting use cases you could apply fine-tuning an ensemble approach.

A lot of people sleep on 7B, but I think Mistral is a little different - there’s a lot of exploring to be had finding these use cases but I think they’re out there waiting to be discovered.

I’ll definitely report back on how the first attempt at fine-tuning this myself goes. Until then, I suppose it would be great for any roleplay or basic chat interaction. Given it’s low headroom - it’s much more lightweight to prototype with outside of the other families and model sizes.

If anyone else has a particular use case for 7B models - let us know here. Curious to know what others are doing with smaller params.

Thank you for the kind words. I really appreciate your comment, and I could not agree more. I believe the junction of FOSS and AI will be integral to a future we are starting to see emerge. I started this community because I feel this is important. It means a lot to me others feel the same, you included.

And hey - you should give yourself more credit. I want you (and everyone else reading this now) to know I started my career in technology as a gamer. Nothing more, nothing less. I could not afford access to a higher education, so my first jobs were in fast food, retail, and warehouses. It took a few opportunities before getting a foot in the door, but I did that through teaching myself anything I found interesting through online youtube tutorials, random forums, video game analogies, and building PCs with friends and family. If I can do it, I know you can too! Progress is non-linear. Have hope for yourself.

AI is going to accelerate us to a future where anyone can learn just as much as I have in half the time (if not less). I encourage you to lift yourself up and get excited! A whole new world of groundbreaking tools and applications will be available to you to develop yourself in all kinds of ways imaginable.

Whatever it is you're looking for, I hope you find it! Let me know if I can ever improve in how I help you seek it with FOSAI.

This is really interesting, thanks for sharing!

For our hardware heads, do you think you could share a spec sheet of your device?

Love seeing 70B parameter models being ran at home! Imagine where we’ll be a year from now..

After finally having a chance to test some of the new Llama-2 models, I think you're right. There's still some work to be done to get them tuned up... I'm going to dust off some of my notes and get a new index of those other popular gen-1 models out there later this week.

I'm very curious to try out some of these docker images, too. Thanks for sharing those! I'll check them when I can. I could also make a post about them if you feel like featuring some of your work. Just let me know!

FWIW, it's a new term I am trying to coin in FOSS communities (Free, Open-Source Software communities). It's a spin off of 'FOSS', but for AI.

There's literally nothing wrong with FOSS as an acronym, I just wanted to use one more focused in regards to AI tech to set the right expectations for everything shared in /c/FOSAI

I felt it was a term worth coining given the varied requirements and dependancies AI/LLMs tend to have compared to typical FOSS stacks. Making this differentiation is important in some of the semantics these conversations carry.

The good news is that it appears AMD is aware of this and has partnered with HuggingFace to make the hardware side of things more accessible.

While they're still planning what that might look like - having another contender in the space other than NVIDIA's CUDA architecture is healthy for everyone in the open-source communities. I'm hoping with their help we can continue to reduce the cost running FOSAI software (even if NVIDIA GPUs continue to outperform).

I think initiatives like this - and the reduction of costs in training due to algorithm optimizations will carry us to the promised land of true AGI that is both low-compute and widely available. We're still in the early phase, though.

Curious to see how all of this continues to develop the rest of this year...

Lol, you had me in the first half not gonna lie. Well done, you almost fooled me!

Glad you had some fun! gpt4all is by far the easiest to get going with imo.

I suggest trying any of the GGML models if you haven't already! They outperform almost every other model format at the moment.

If you're looking for more models, TheBloke and KoboldAI are doing a ton for the community in this regard. Eric Hartford, too. Although TheBloke is typically the one who converts these into more accessible formats for the masses.

Great suggestions! I've actually never interfaced with that first channel (SECourses). Looks like some solid tutorials. Definitely going to check that out. Thanks for sharing!

In my opinion writing a paper is good practice no matter the results. It might help you discern more valuable insights from your testing or approach.

In this situation, you have almost nothing to lose! I say go for it. Do both. Start a paper draft now and iterate upon it as you benchmark more results. Often times writing and reflecting on your own research reinforces some of the concepts you're tackling. All the more reason to write something up, even if you don't release it.

If you do end up writing one, be sure to share it here!

Great question. I ponder this too, which is why I started /c/FOSAI. We have to do everything we can to make sure our future stays open for all, our faith cannot be put into the hands of a select few, but rather - the majority of many.

Time will tell who truly supports this. I'm hopeful OpenAI is the good guy we want them to be, but other businesses keep me from jumping to that conclusion. I like what they are doing alongside Microsoft, but we need more players in the game. Fresh minds to shake things up a little.

If you're reading this, support FOSS, support FOSAI, and support the Fediverse. It's the only way we can take back the internet, one server at a time.

Absolutely! I'm having a blast launching /c/FOSAI over at Lemmy.world. I'll do my best to consistently cross-post for everyone over here too!

🤗

I believe it's a different technique (at least far as I understand the topics).

According to Mosaic, MPT (i.e. MPT-7B-StoryWriter-65k+) uses a different underlying architecture which enables their long context lengths.

The original author of this new method (SuperHOT by kaiokendev) shares what he has learned about this method here:

• https://kaiokendev.github.io/til
• https://kaiokendev.github.io/context

Great suggestion! I'll add that to the list.

For anyone unsure what this is, here's a quick peek at Comfy-ui.

Kind of similar to ControlNet, it's another method for expanding the options you have over your Stable Diffusion workflow.