Image by editor
, Introduction
ChatGPT, Cloud, Gemini. You know the name. But here’s a question: what if you run your own model instead? This sounds ambitious. it. You can deploy a task big language model (LLM) in less than 10 minutes without spending a single dollar.
This article breaks it down. First, we’ll figure out what you really need. Then we’ll look at the actual costs. Finally, we’ll deploy TinyLama to Hugging Face for free.
Before launching your model, you probably have a lot of questions. For example, what tasks am I expecting my model to perform?
Let’s try to answer this question. If you need a bot for 50 users, you don’t need GPT-5. Or if you plan to perform sentiment analysis on 1,200+ tweets a day, you may not need a model with 50 billion parameters.
Let’s first take a look at some popular use cases and the models that can perform those functions.
As you can see, we matched the model to the task. This is what you should do before starting.
, Breaking down the real cost of hosting an LLM
Now that you know what you need, let me show you how much it costs. Hosting a model isn’t just about the model; It’s also about where the model runs, how often it runs, and how many people interact with it. Let’s decode the real costs.
, Calculate: The biggest costs you will face
if you drive one Central processing unit (CPU) on 24/7 Amazon Web Services (AWS) EC2, which will cost around $36 per month. However, if you run a graphics processing unit (GPU) for example, it will cost about $380 per month – 10 times the cost. So be careful when calculating the cost of your large language model, as this is the main expense.
(Calculations are approximate; to see the actual price, please visit: AWS EC2 Pricing,
, Storage: Small cost, unless your model is huge
Let’s roughly calculate disk space. A 7B (7 billion parameters) model takes about 14 gigabyte (GB). Cloud storage costs around $0.023 per GB per month. So the difference between the 1GB model and the 14GB model is about $0.30 per month. Storage costs may be negligible if you do not plan to host a 300B parameter model.
, Bandwidth: Cheap until you get bigger
Bandwidth is important when your data moves, and when other people use your model, so does your data. AWS charges $0.09 per GB after the first GB, so you’re looking at pennies. But if you scale to millions of requests, you should calculate this carefully too.
(Calculations are approximate; to see the actual price, please visit: AWS Data Transfer Pricing,
, Free Hosting Options You Can Use Today
embracing face space Allows you to host small models with CPU for free. render And Railway Offer free tiers that work for low-traffic demos. If you’re experimenting or building a proof of concept, you can get pretty far without spending a dime.
, Choose a model you can actually drive
Now we know the cost, but which model should you drive? Of course, each model has its advantages and disadvantages. For example, if you download a 100-billion-parameter model onto your laptop, I guarantee it won’t work unless you have a top-tier, purpose-built workstation.
Let’s look at the different models available on Hugging Face so that you can play them for free, as we are going to do in the next section.
tinyllama: This model requires no setup and runs using the free CPU tier on Hugging Face. It is designed for simple conversation tasks, answering simple questions, and text generation.
It can be used to quickly build and test chatbots, run quick automation experiments, or create internal question-answering systems for testing before expanding into infrastructure investments.
DistillGPT-2:It is fast and light also. This makes it perfect for hugging face space. Fine for text completion, very simple classification tasks or short responses. This is appropriate to understand how LLMs function without resource constraints.
fi-2: A small model developed by Microsoft which proves to be quite effective. It still runs on the free tier from Hugging Face but offers better logic and code generation. Employ it for natural language-to-SQL query generation, simple Python code completion, or customer review sentiment analysis.
flan-t5-small: This is Google’s instruction-tuning model. Designed to respond to commands and provide answers. Useful for generation when you want deterministic output on free hosting, such as summarization, translation, or question-and-answer.
, Deploy TinyLlama in 5 minutes
Let’s build and deploy TinyLama using Hugging Face Space for free. No credit card, no AWS account, no Docker headaches. Just a working chatbot that you can share with a link.
, Step 1: Go to Hugging Face Space
go towards huggingface.co/spaces And click on “New Location” like the screenshot below.
Name the location whatever you want and add a short description.
You can leave other settings as they are.
Click “Create Space”.
, Step 2: Write App.py
Now, click on “create the app.py” from the bottom screen.
Paste the below code inside this App.py.
This code loads TinyLlama (with the build files available on Hugging Face), wraps it in a chat function, and uses Gradio To create a web interface. chat() The method formats your message correctly, generates a response (up to a maximum of 100 tokens), and returns only answers from the model for the question you asked (this does not include duplication).
Here This is the page where you can learn how to write code for any Hugging Face model.
Let’s look at the code.
import gradio as gr
from transformers import AutoModelForCausalLM, AutoTokenizer
model_name = "TinyLlama/TinyLlama-1.1B-Chat-v1.0"
tokenizer = AutoTokenizer.from_pretrained(model_name)
model = AutoModelForCausalLM.from_pretrained(model_name)
def chat(message, history):
# Prepare the prompt in Chat format
prompt = f"<|user|>n{message}n<|assistant|>n"
inputs = tokenizer(prompt, return_tensors="pt")
outputs = model.generate(
**inputs,
max_new_tokens=100,
temperature=0.7,
do_sample=True,
pad_token_id=tokenizer.eos_token_id
)
response = tokenizer.decode(outputs(0)(inputs('input_ids').shape(1):), skip_special_tokens=True)
return response
demo = gr.ChatInterface(chat)
demo.launch()After pasting the code, click on “Commit the new file to main”. Please see the screenshot below as an example.
Hugging Face will automatically detect it, install the dependencies, and deploy your app.
In the meantime, create a requirements.txt File or you will get an error like this.
, Step 3: Create Requirements.txt
Click “Files” in the upper right corner of the screen.
Here, click on “Create a new file” like the screenshot below.
Name the file “requirements.txt” and add 3 Python libraries, as shown in the following screenshot (transformers, torch, gradio,
transformer Here the model is loaded and deals with tokenization. torch The model runs because it provides the neural network engine. Gradio creates a simple web interface so users can chat with models.
, Step 4: Run and test your deployed model
When you see the green light “Running”, it means you’re done.
Let’s test it now.
You can test it by first clicking on the app here.
Let’s use this to write a Python script that detects outliers comma separated values (CSV) file using z-score and interquartile range (IQR).
Here are the test results;
, Understanding the deployment you just created
The result is that you are now able to spin up 1B+ parameter language models and never need to touch a terminal, set up a server, or spend a dollar. Hugging Face takes care of hosting, compute, and scaling (to an extent). Paid tier available for more traffic. But for experimental purposes, this is ideal.
Best way to learn? Deploy first, optimize later.
, Where to go next: Improve and extend your model
You now have a working chatbot. But TinyLama is just the beginning. If you need better responses, try upgrading to a Phi-2 or Mistral 7B using the same process. just change the model name app.py And add a little more computing power.
For faster responses, look into quantization. You can also connect your model to a database, add memory to the conversation, or fine-tune it to your data, so the only limit is your imagination.
Nate Rosidi Is a data scientist and is into product strategy. He is also an adjunct professor teaching analytics, and is the founder of StratScratch, a platform that helps data scientists prepare for their interviews with real interview questions from top companies. Nate writes on the latest trends in the career market, gives interview advice, shares data science projects, and covers everything SQL.
