Object Detection with Hugging Face Transformers

How to use HuggingFace Transformers to build your own object detection
Author

Kedar Dabhadkar

Published

January 3, 2021

How to use HuggingFace Transformers to build your own object detection

# !pip install transformers==4.6.1
# !pip install sentencepiece
# !pip install protobuf
from transformers import AutoFeatureExtractor, AutoModelForObjectDetection

extractor = AutoFeatureExtractor.from_pretrained("facebook/detr-resnet-50")
model = AutoModelForObjectDetection.from_pretrained("facebook/detr-resnet-50")
---------------------------------------------------------------------------
ModuleNotFoundError                       Traceback (most recent call last)
~\AppData\Local\Temp/ipykernel_17336/1239257601.py in <module>
----> 1 from transformers import AutoFeatureExtractor, AutoModelForObjectDetection
      2 
      3 extractor = AutoFeatureExtractor.from_pretrained("facebook/detr-resnet-50")
      4 model = AutoModelForObjectDetection.from_pretrained("facebook/detr-resnet-50")

ModuleNotFoundError: No module named 'transformers'

Write evaluation functions

# Write a wrapper function to generate a Q&A pipeline from any specified model
def transformer_models(model_name):
    model =  AutoModelForQuestionAnswering.from_pretrained(model_name)
    tokenizer = AutoTokenizer.from_pretrained(model_name)
    qa_pipeline = pipeline('question-answering', model=model, tokenizer=tokenizer)
    return qa_pipeline


# Q&A using the default DistilBERT model
def distilbert_answer(context, query):
    '''
    Evaluate the specified context and query using DistilBERT
    '''
    distilbert_model = pipeline('question-answering')
    try:
        res = distilbert_model(question=query, context=context)
        return res['answer']
    except:
        return "Snap! The model couldn't find an answer. Try a different query."

    
# Q&A using the default RoBERTa model
def roberta_answer(context, query):
    '''
    Evaluate the specified context and query using RoBERTa
    '''
    roberta_model = transformer_models("deepset/roberta-base-squad2")
    try:
        res = roberta_model(question=query, context=context)
        return res['answer']
    except:
        return "Snap! The model couldn't find an answer. Try a different query."

    
# Q&A using the default ALBERT model
def albert_answer(context, query):
    '''
    Evaluate the specified context and query using ALBERT
    '''
    albert_model = transformer_models("twmkn9/albert-base-v2-squad2")
    try:
        res = albert_model(question=query, context=context)
        return res['answer']
    except:
        return "Snap! The model couldn't find an answer. Try a different query."

Input passage

context = """Let me tell you how you can play the money beat on your drums. To get this beat going start 
with one part at a time. This way you can part your mind, and feel the groove a lot better. With your hi hat, 
play constant eight notes. We will add in some accents in the future, but for now, just play eight notes. 
Remember to count out loud when you are playing, it will help you out a lot!

Now that you have got this, try to not think about it. When people first learn to play the drums they usually 
think too much. This is where they start to make mistakes, they overthink. Your hi hat hand will not change 
this motion, so try to forget about it. Now it's time to concentrate on your other hand. With this hand, you will 
be playing quarter notes on the snare. These snare hits will be on the 2 and 4 count.

Good! Now let’s finish it off with the bass drum. This too will be playing quarter notes, however, not on the 
2 and four. Most beginners will have trouble with this, they will end up playing their snare and bass drum at 
the same time. Take your time and it will come to you. Play the bass on 1 and 3 counts."""
context = context.replace("\n", " ")

Ask a question

query = "What is this passage about?"
# ALBERT
albert_answer(context, query)
' how you can play the money beat on your drums.'
# DistilBERT
distilbert_answer(context, query)
'where they start to make mistakes'
# RoBERTa
roberta_answer(context, query)
'money beat'

Let’s compare answers for various questions

queries = ["What is this passage about?",
          "How can one play the money beat?",
          "Where does on play the quarter notes?",
          "Where does one play the bass?",
          "What mistakes do beginners make?"]
model_names = ["DistilBERT", "RoBERTa", "ALBERT"]
distilbert_answers, roberta_answers, albert_answers = [], [], []
for query in queries:
    distilbert_answers.append(distilbert_answer(context, query))
    roberta_answers.append(roberta_answer(context, query))
    albert_answers.append(albert_answer(context, query))
results = pd.DataFrame()
results['Query'] = queries
results['DistilBERT'] = distilbert_answers
results['RoBERTa'] = roberta_answers
results['ALBERT'] = albert_answers
results
Query DistilBERT RoBERTa ALBERT
0 What is this passage about? where they start to make mistakes money beat how you can play the money beat on your drums.
1 How can one play the money beat? on your drums start with one part at a time start with one part at a time.
2 Where does on play the quarter notes? the snare on the snare on the snare.
3 Where does one play the bass? 1 and 3 counts 1 and 3 counts 1 and 3 counts.
4 What mistakes do beginners make? snare and bass drum they overthink playing their snare and bass drum at the sam...

Read more

[1] HuggingFace Tranformers pipeline: https://huggingface.co/transformers/main_classes/pipelines.html#transformers.QuestionAnsweringPipeline
[2] DiltilBERT: https://huggingface.co/transformers/model_doc/distilbert.html
[3] RoBERTa: https://huggingface.co/transformers/model_doc/roberta.html
[4] ALBERT: https://huggingface.co/transformers/model_doc/albert.html