LM Studio: An Easy Way to Use Open Source Models

Had some great discussions at a recent RMAIIG meeting with some fellow attendees about open source models and how many people find them "cumbersome" since solutions like ChatGPT have an easy-to-use UI. Surprisingly, not many people have heard of today's topic of discussion: LM Studio. So, I'd rather bring some awareness to the solution, because it is kinda neat.

LM Studio is an application made to handle chatbot-like queries to open source models. What kind of models, you ask? Literally any GGUF model on Hugging Face. Find one you like, grab the link, and LM Studio will get it set up automatically so all you have to do is query your model.

We'll talk about setup and some common activities you might want to do on LM Studio in this post to help people who want to be more holistically mindful of their AI usage by utilizing and supporting open-source.

Later this week, I also hope to highlight some lesser known open source models, since there are many originating outside of the U.S. that are truly innovative in how accessible they're making this technology. Be on the lookout for that!

For now though, let's dive in.

Why Use LM Studio (and open source models)?

I am obviously very for the accessibility that LM Studio is trying to bring into the AI space, since open-source technology can sometimes have a barrier to adoption. Here's some reasons I've identified to consider LM Studio in your day-to-day workflows:

1. Privacy - All of these models run locally on your machine, and unless it's either compromised by a bad actor or simply being an LLM, all the data stays on your device.

2. Energy Efficiency - Many of these models are insanely low energy cost to run per query / token. You can read more about energy drain from models in a recent post of mine on AI and the environment.

3. Open Source - There's inherent transparency as to what is happening under the hood with open-source technology, even if you may not understand the tech pieces. People that do understand tend to be very vocal about issues. Plus, if you do know some programming, you can contribute to these projects directly. It often is a volunteer effort, so consider donating to open-source tech you enjoy.

4. Simple UI - The primary method of interacting with this model should be near identical to how ChatGPT, Perplexity, Claude, and other models structure their windows, so it's an easy transition.

One thing I would like to highlight is that LLMs can tend to be somewhat RAM-hungry on your computer. If you're using an older device with less resources available, I (and LM Studio themselves) would recommend sticking to a lower-parameter / size model and smaller context sizes, which we'll cover later.

Getting Started

Navigate to LMStudio's website and pick out the version for your operating system. Keep in mind the system requirements for LM Studio, especially on Mac. Right now, LM Studio doesn't support Intel-chip Macbooks, so that's a potential issue for people with older hardware.

Go ahead and install LM Studio. Once that's finished, the solution will guide you on getting your first model installed, usually something like Llama or Mistral. For me, I think it was Llama 3.2 3B Instruct. It's built and maintained by Meta, so keep that in mind.

And that's it! Once the model is downloaded, you can open a chat with the plus icon on the top left, and start using it just like other chatbot-like experiences you're used to.

Quick Tips

When you're creating a chat, by default there won't be any models loaded. You'll need to hit the 'Load Model' button in the chat window to select whatever models you have installed.

Loading a model comes with some additional parameters to set. There will be preset options, however you can tweak as needed. Particularly for users with older or less power, note the context length slider (shown below). A lower context length means less memory overhead, also meaning less resource usage on your computer, to provide a response.

Models outside of the 'Staff Picks' may not be fully compatible with LM Studio, but most serve much of the same purpose you might want to use cloud-hosted models for. You can search for models in the 'Discover' tab of LM Studio.

Keep in mind that not all models can do everything, and it's worth poking around Hugging Face to find one that fits what you need. For example, you may want to use text-to-image capabilities on your device. A model like this Flux GGUF Conversion may work out for this purpose. Going back to incompatibility, you may end up with a couple of duds trying to find the right one. That's the fun of having the choice of what model you use!

Conclusion

LM Studio is a great tool, and I'm not even getting into the developer stuff you can do with it (like hooking in local models in place of OpenAI in Python!) that make it exciting for a guy like me. Highly recommend just playing around and testing to see what models work for you.

Please reach out if you're interested in talking more about LM Studio! Later this week, I'm covering more on open source models you probably haven't heard of that can be used with LM Studio. See you then!

Today's Cha Cha photo comes from a long day of sulking about the house. Nothing better than cuddles with dad!