How one can Construct a Calorie Advisor App Utilizing GenAI?

Introduction

Synthetic Intelligence has many use instances, and among the finest ones are within the Well being Trade. It might probably actually assist individuals preserve a more healthy life. With the growing increase in generative AI, sure functions are made nowadays with much less complexity. One very helpful utility that may be constructed is the Calorie Advisor App. On this article, we’ll solely have a look at this, impressed by caring for our well being. We will likely be constructing a easy Calorie Advisor App the place we will enter the pictures of the meals, and the app will assist us calculate the energy of every merchandise current within the meals. This mission is part of NutriGen, specializing in well being by means of AI.

Studying Goal

• The App we will likely be creating on this article will likely be based mostly on primary Immediate engineering and picture processing strategies.
• We will likely be utilizing Google Gemini Professional Imaginative and prescient API for our use case.
• Then, we’ll create the code’s construction, the place we’ll carry out Picture Processing and Immediate Engineering. Lastly, we’ll work on the Person Interface utilizing Streamlit.
• After that, we’ll deploy our app to the Hugging Face Platform for Free.
• We can even see among the issues we’ll face within the output the place Gemini fails to depict a meals merchandise and offers the mistaken calorie depend for that meals.  We can even focus on completely different options for this downside.

Pre-Requisites

Let’s begin with implementing our mission, however earlier than that, please guarantee you’ve a primary understanding of generative AI and LLMs. It’s okay if you understand little or no as a result of, on this article, we will likely be implementing issues from scratch.

For Important Python Immediate Engineering, a primary understanding of Generative AI and familiarity with Google Gemini is required. Moreover, primary data of Streamlit, Github, and Hugging Face libraries is critical. Familiarity with libraries reminiscent of PIL for picture preprocessing functions can be useful.

This text was revealed as part of the Information Science Blogathon.

Venture Pipeline

On this article, we will likely be engaged on constructing an AI assistant who assists nutritionists and people in making knowledgeable selections about their meals selections and sustaining a wholesome life-style.

The circulate will likely be like this: enter picture -> picture processing -> immediate engineering -> ultimate operate calling to get the output of the enter picture of the meals. It is a transient overview of how we’ll method this downside assertion.

Overview of Gemini Professional Imaginative and prescient

Gemini Professional is a multimodal LLM constructed by Google. It was educated to be multimodal from the bottom up. It might probably carry out properly on varied duties, together with picture captioning, classification, summarisation, question-answering, and so forth. One of many fascinating information about it’s that it makes use of our well-known Transformer Decoder Structure. It was educated on a number of sorts of knowledge, lowering the complexity of fixing multimodal inputs and offering high quality outputs.

Step1: Creating the Digital Atmosphere

Making a digital surroundings is an efficient follow to isolate our mission and its dependencies such that they don’t coincide with others, and we will at all times have completely different variations of libraries we’d like in numerous digital environments. So, we’ll create a digital surroundings for the mission now. To do that, observe the talked about steps under:

• Create an Empty folder on the desktop for the mission.
• Open this folder in VS Code.
• Open the terminal.

Write the next command:

pip set up virtualenv
python -m venv genai_project

You need to use the next command should you’re getting sa et execution coverage error:

Set-ExecutionPolicy RemoteSigned -Scope Course of

Now we have to activate our digital surroundings, for that use the next command:

.genai_projectScriptsactivate

We’ve got efficiently created our digital surroundings.

Step Create Digital Atmosphere in Google Colab

We will additionally create our Digital Atmosphere in Google Colab; right here’s the step-by-step process to do this:

• Create a New Colab Pocket book
• Use the under instructions step-by-step

!which python
!python --version
#to test if python is put in or not

%env PYTHONPATH=
# setting python path surroundings variable in empty worth making certain that python
# will not seek for modules and packages in further listing. It helps
# in avoiding conflicts or unintended module loading.

!pip set up virtualenv 

# create digital surroundings 
!virtualenv genai_project

!wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh

#It will assist obtain the miniconda installer script which is used to create
# and handle digital environments in python

!chmod +x Miniconda3-latest-Linux-x86_64.sh
# this command is making our mini conda installer script executable inside
# the colab surroundings. 

!./Miniconda3-latest-Linux-x86_64.sh -b -f -p /usr/native
# that is used to run miniconda installer script and 
# specify the trail the place miniconda needs to be put in

!conda set up -q -y --prefix /usr/native python=3.8 ujson
#it will assist set up ujson and python 3.8 set up in our venv.

import sys
sys.path.append('/usr/native/lib/python3.8/site-packages/')
#it is going to enable python to find and import modules from a venv listing

import os
os.environ['CONDA_PREFIX'] = '/usr/native/envs/myenv'

# used to activate miniconda enviornment 

!python --version
#checks the model of python throughout the activated miniconda surroundings

Therefore, we additionally created our digital surroundings in Google Colab. Now, let’s test and see how we will make a primary .py file there.

!supply myenv/bin/activate
#activating the digital surroundings

!echo "print('Howdy, world!')" >> my_script.py
# writing code utilizing echo and saving this code in my_script.py file

!python my_script.py
#operating my_script.py file

It will print Howdy World for us within the output. So, that’s it. That was all about working with Digital Environments in Google Colab. Now, let’s proceed with the mission.

Step2: Importing Crucial Libraries

import streamlit as st
import google.generativeaias genai 
import os 
from dotenv import load_dotenv
load_dotenv()
from PIL import Picture

If you’re having bother importing any of the above libraries, you possibly can at all times use the command “pip set up library_name” to put in it.

We’re utilizing the Streamlit library to create the essential consumer interface. The consumer will have the ability to add a picture and get the outputs based mostly on that picture.

We use Google Generative to get the LLM and analyze the picture to get the calorie depend item-wise in our meals.

Picture is getting used to carry out some primary picture preprocessing.

Step3: Organising the API Key

Create a brand new .env file in the identical listing and retailer your API key. You may get the Google Gemini API key from Google MakerSuite.

Step4: Response Generator Perform

Right here, we’ll create a response generator operate. Let’s break it down step-by-step:

Firstly, we used genes. Configure to configure the API we created from the Google MakerSuite Web site. Then, we made the operate get_gemini_response, which takes in 2 enter parameters: the enter immediate and the picture. That is the first operate that can return the output in textual content.

genai.configure(api_key=os.getenv("GOOGLE_API_KEY"))

def get_gemini_response(input_prompt, picture):

    mannequin = genai.GenerativeModel('gemini-pro-vision')

    response = mannequin.generate_content([input_prompt, image[0]])

    return response

Right here, we’re utilizing the ‘Gemini-pro-vision’ mannequin as a result of it’s multimodal. After calling our mannequin from the genie.GenerativeModel dependency, we’re simply passing in our immediate and the picture knowledge to the mannequin. Lastly, based mostly on the directions supplied within the immediate and the picture knowledge we fed, the mannequin will return the output within the type of textual content that represents the calorie depend of various meals gadgets current within the picture.

Step5: Picture Preprocessing

This operate checks if the uploaded_file parameter is None, that means the consumer has uploaded a file. If a file has been uploaded, the code proceeds to learn the file content material into bytes utilizing the getvalue() methodology of the uploaded_file object. It will return the uploaded file’s uncooked bytes.

The bytes knowledge obtained from the uploaded file is saved in a dictionary format underneath the key-value pair “mime_type” and “knowledge.” The “mime_type” key shops the uploaded file’s MIME kind, which signifies the kind of content material (e.g., picture/jpeg, picture/png). The “knowledge” key shops the uploaded file’s uncooked bytes.

The picture knowledge is then saved in an inventory named image_parts, which accommodates a dictionary with the uploaded file’s MIME kind and knowledge.

def input_image_setup(uploaded_file):
    if uploaded_file isnotNone:
        #Learn the file into bytes
        bytes_data = uploaded_file.getvalue()
        image_parts = [
            
                "mime_type":uploaded_file.type, 
                "data":bytes_data
            
        ]
        return image_parts
    else:
        elevate FileNotFoundError("No file uploaded")

Step6: Creating the UI

So, lastly, it’s time to create the consumer interface for our mission. As talked about earlier than, we will likely be utilizing the Streamlit library to put in writing the code for the entrance finish.

## initialising the streamlit app
st.set_page_config(page_title="Energy Advisor App")
st.header("Energy Advisor App")
uploaded_file = st.file_uploader("Select a picture...", kind=["jpg", "jpeg", "png"])
picture = ""
if uploaded_file isnotNone:
    picture = Picture.open(uploaded_file)
    st.picture(picture, caption="Uploaded Picture", use_column_width=True)
submit = st.button("Inform me concerning the complete energy")

Initially, we arrange the web page configuration utilizing set_page_config and gave the app a title. Then, we created a header and added a file uploader field the place customers can add photographs. St. Picture exhibits the picture that the consumer uploaded to the UI. Eventually, there’s a submit button, after which we’ll get the outputs from our massive language mannequin, Gemini Professional Imaginative and prescient.

Step7: Writing the System Immediate

Now’s the time to be inventive. Right here, we’ll create our enter immediate, asking the mannequin to behave as an professional nutritionist. It’s not crucial to make use of the immediate under; you may also present your customized immediate. We’re asking our mannequin to behave a sure method for now. Primarily based on the enter picture of the meals supplied, we’re asking our mannequin to learn that picture knowledge and generate the output, which can give us the calorie depend of the meals gadgets current within the picture and supply a judgment of whether or not the meals is wholesome or unhealthy. If the meals is dangerous, we ask it to offer extra nutritious alternate options to the meals gadgets in our picture. You may customise it extra in line with your wants and get a superb approach to maintain observe of your well being.

Generally it won’t in a position to learn the picture knowledge correctly, we’ll focus on options concerning this additionally on the finish of this text.

input_prompt = """

You might be an professional nutritionist the place you have to see the meals gadgets from the 
picture and calculate the full energy, additionally give the small print of all 
the meals gadgets with their respective calorie depend within the under fomat.

        1. Merchandise 1 - no of energy

        2. Merchandise 2 - no of energy

        ----

        ----

Lastly you may also point out whether or not the meals is wholesome or not and likewise point out 
the proportion break up ratio of carbohydrates, fat, fibers, sugar, protein and 
different essential issues required in our food plan. For those who discover that meals isn't wholesome 
then you should present some different wholesome meals gadgets that consumer can have 
in food plan.

"""
if submit:

    image_data = input_image_setup(uploaded_file)

    response = get_gemini_response(input_prompt, image_data)

    st.header("The Response is: ")

    st.write(response)

Lastly, we’re checking that if the consumer clicks the Submit button, we’ll get the picture knowledge from the

input_image_setup operate we created earlier. Then, we move our enter immediate and this picture knowledge to the get_gemini_response operate we created earlier.  We name all of the capabilities we created earlier to get the ultimate output saved in response.

Step8: Deploying the App on Hugging Face

Now’s the time for deployment. Let’s start.

Will clarify the best approach to deploy this app that we created. There are two choices that we will look into if we wish to deploy our app: one is Streamlit Share, and the opposite one is Hugging Face. Right here, we’ll use Hugging Face for the deployment; you possibly can strive exploring deployment on Streamlit Share iFaceu in order for you. Right here’s the reference hyperlink for that – Deployment on Streamlit Share

First, let’s shortly create the necessities.txt file we’d like for the deployment.

Open the terminal and run the under command to create a necessities.txt file.

pip freeze > necessities.txt1plainText

It will create a brand new textual content file named necessities. All of the mission dependencies will likely be accessible there. If this causes an error, it’s okay. You may at all times create a brand new textual content file in your working listing and replica and paste the necessities.txt file from the GitHub hyperlink I’ll present subsequent.

Now, just be sure you have these information helpful (as a result of that’s what we’d like for the deployment):

• app.py
• .env (for the API credentials)
• necessities.txt

For those who don’t have one, take all these information and create an account on the cuddling face. Then, create a brand new house and add the information there. That’s all. Your app will likely be routinely deployed this manner. Additionally, you will have the ability to see how the deployment is happening in real-time. If some error happens, you possibly can at all times determine it out with the straightforward interface and, in fact, the cuddling face group, which has a number of content material on resolving some frequent bugs throughout deployment.

After a while, it is possible for you to to see the app working. Woo hoo! We’ve got lastly created and deployed our calorie predictor app. Congratulations!!, You may share the working hyperlink of the app with the family and friends you simply constructed.

Right here’s the working hyperlink to the app that we simply created – The Alorcalorieisor App

Let’s take a look at our app by offering an enter picture to it:

Earlier than:

After:

Full Venture GitHub Hyperlink

Right here’s the whole github repository hyperlink that features supply code and different useful data concerning the mission.

You may clone the repository and customise it in line with your necessities. Attempt to be extra inventive and clear in your immediate, as it will give your mannequin extra energy to generate appropriate and correct outputs.

Scope of Enchancment

Issues that may happen within the outputs generated by the mannequin and their options:

Generally, there could possibly be conditions the place you’ll not get the proper output from the mannequin. This will likely occur as a result of the mannequin was not in a position to predict the picture accurately. For instance, should you give enter photographs of your meals and your meals merchandise accommodates pickles, then our mannequin may take into account it one thing else. That is the first concern right here.

• One approach to deal with that is by means of efficient immediate engineering strategies, like few-shot immediate engineering, the place you possibly can feed the mannequin with examples, after which it is going to generate the outputs based mostly on the learnings from these examples and the immediate you supplied.
• One other resolution that may be thought of right here is creating our customized knowledge and fine-tuning it. We will create knowledge containing a picture of the meals merchandise in a single column and an outline of the meals gadgets current within the different column. It will assist our mannequin study the underlying patterns and predict the gadgets accurately within the picture supplied. Thus, getting extra appropriate outputs of the calorie depend for the images of the meals is important.
• We will take it additional by asking the consumer about his/her vitamin objectives and asking the mannequin to generate outputs based mostly on that. (This fashion, we will tailor the outputs generated by the mannequin and provides extra user-specific outputs.)

Conclusion

We’ve delved into the sensible utility of Generative AI in healthcare, specializing in the creation of the Calorie Advisor App. This mission showcases the potential of AI to help people in making knowledgeable selections about their meals selections and sustaining a wholesome life-style. From organising the environment to implementing picture processing and immediate engineering strategies, we’ve coated the important steps. The app’s deployment on Hugging Face demonstrates its accessibility to a wider viewers. Challenges like picture recognition inaccuracies have been addressed with options reminiscent of efficient immediate engineering. As we conclude, the Calorie Advisor App stands as a testomony to the transformative energy of Generative AI in selling well-being.

Key Takeaways

• We’ve got mentioned quite a bit thus far, Beginning with the mission pipeline after which a primary introduction to the big language mannequin Gemini Professional Imaginative and prescient.
• Then, we began with the hands-on implementation. We created our digital surroundings and API key from Google MakerSuite.
• Then, we carried out all our coding within the created digital surroundings. Additional, we mentioned tips on how to deploy the app on a number of platforms, reminiscent of Hugging Face and Streamlit Share.
• Other than that, we thought of the potential issues that may happen, and mentioned soluFaces to these issues.
•  Therefore, it was enjoyable engaged on this mission. Thanks for staying until the top of this text; I hope you bought to study one thing new.

Often Requested Questions

The media proven on this article isn’t owned by Analytics Vidhya and is used on the Creator’s discretion.