In this tutorial, we’ll go through how to use a deep learning algorithm that predicts the make and model of a car from a picture we take on Android.
We’ll use the native Camera2 API to take a picture of a car, then we’ll use the Car Make and Model algorithm on Algorithmia to process the image in order to display the car’s make and model along with other metadata.
You can either clone the repo directly and follow along with the README instructions to see the fully complete app, or you can follow the tutorial below to build an Android app that utilizes deep learning.
First things first, let’s create a new app in Android Studio.
Select New Project and follow along with the New Project wizard. You can feel free to give your new app any name you like. Then, select your target devices and when prompted with the Add Activity screen, select “Empty Activity”:
Then name it “MainActivity”. This activity will hold the code that we’ll write to access the camera and save the picture. Then we’ll use File Provider to pass the URI to the next activity which we’ll create now.
To add another activity right click on your project name (this example project is called com.example.carmakemodelapp) and add a new empty activity called “CarRecognitionActivity”.
This activity will be used when we call the algorithm and render the results.
Now before we get started coding, we need to set up the rest of our project.
Settings, Permissions, & API Key
Follow the steps below to make sure you have the dependencies, permissions, and your API key set as needed.
Note that if you forked the repo, you will have everything you need except for you’ll need to add your API key.
Step One: Dependencies
Add the Algorithmia client to your app in the dependencies section of app/build.gradle, like so:
Step Two: Internet Permissions
To ensure that you can access the Internet and the Camera2 API from the app, you’ll specify these permissions in “AndroidManifest.xml”:
Also go ahead and add a blank activity with the name “.CarRecognitionActivity” which we’ll use a bit later on:
Next, because we want to use FileProvider to ensure we are passing around files securely, you’ll need to add this to your “AndroidManifest.xml” at the bottom of your manifest file:
Step Three: API Key
In “strings.xml”, replace the demo API key with your API key, which can be found under the credentials section of your profile on the Algorithmia website. Also make sure the file provider authority string is set to your package name. For more information on File Provider check out the Android Docs.
Make sure you’ve replaced YOUR_API_KEY with the API key under your account so that the Algorithmia client can authenticate!
Next create a folder called “xml” under your “res” folder so the file provider can find your image. Then add a file called “file_provider_paths.xml” and copy and paste the following:
Adding layouts to the app
Here we’ll add some UI elements. In the file “res/layout/activity_main.xml” add the following Layout and Button.
You can also do this through the visual editor, but since we just have a few simple things to add to the layout we’ll add them to the xml:
Now, in the file “res/layout/activity_car_results.xml” add:
Note that if you don’t have those files already under res/layout please add them.
Finally if you don’t have it already, create a file under the drawable folder called “button_colors.xml” and place this in the file:
First, go to “MainActivity.java” and paste in the code below if you haven’t forked the project from GitHub. We’ll go through it chunk by chunk so you understand what’s happening.
Note, that if you haven’t forked the project, you can copy and paste the imports from the repo as we skip them for the sake of brevity in the example below.
Now let’s go through the code.
Directly inside our “MainActivity” class, notice the onCreate method where we initialize our view: “setContentView(R.layout.activity_main)”. This will display our button for taking a picture.
First arg is our application context (MainActivity.this), then the authority that we set in our Manifest file that allows for URI permissions (R.string.file_provider_authority), and finally the timestamped file that we just created.
If we print out the URI we created, it looks like this:
Next is the “takePhoto” method, where we create our first Intent to take an image.
The first argument in our Intent’s “.putExtra” method, is the “MediaStore.EXTRA_OUTPUT”, while the second argument is the URI. This is because we don’t want to save the thumbnail of the image, but instead we want to use a content-resolver URI to store the data. That way we can later pass it into our algorithm.
Now we use “startActivityForResult” which takes our Intent and our request code that we set to 0 in the beginning of our code sample.
Finally we’ll look at the “onActivityResult” method where we create another Intent that references our “CarRecognitionActivity” Activity. Here we’ll set the data for this Intent to our image URI we created in “getOutputMediaFileUri()” and start the activity.
Now we need to look at our next Activity that we created when setting up our app called: “CarRecognitionActivity”:
Now let’s go through the code above starting with the “onCreate” method. Again we are setting our view with “setContentView()”, but this time we are passing in the view from “activity_car_results.xml” that has a new button saying “Find Make and Model of Car” which we need to call our algorithm.
Next, we create a new Intent to tell our app we are about to perform some operation, which in our case, is to get our data from the content provider with ContentResolver.
Next we need to transform the bitmap image into a byteArray. Then, because our algorithm takes only URL’s we’ll be using Algorithmia’s hosted data to upload our image to Algorithmia to get the URL to pass into the Car Make and Model algorithm.
Remember to replace “YOUR_DATA_COLLECTION” in DataDirectory imageDir = client.dir("data://.my/YOUR_DATA_COLLECTION"); with your own data collection name from one of our data sources.
Now we upload our byteArray to our data collection in a file called “myImage.jpg” and then immediately download the file path. Then we call the algorithm and use the “pipe()” method to pass in our data URL to the Car Make and Model algorithm. The reason why we do this is because algorithms don’t accept data uploaded from just any data source like a local file.
In “onPostExecute()” we’ll handle the algorithm response. All we are doing here is handling a null case and when we get an “AlgoSuccess” response, we get that as a JSON string and set the view’s text to show the algorithm’s response. Then, we have some error handling in case our algorithm call isn’t successful!
And here is the result when we run our app, take a picture and run the algorithma: