jeremy(orme)

Last time we made our web app more robust by testing edge cases and fixing bugs. This time we'll refine the ingredient data to improve performance and accuracy. There are a couple of issues with our ingredient database that can reduce the quality and

Last time we got to the point of having an app that aggregates nutritional values for some ingredients and displays them as a % of the recommended daily amounts for the user given their age and gender. This time we'll take a critical look at

404 Apparently TSO Host got fed up of hosting my site and deleted all of my data. I was able to recover almost everything from a recent backup but sadly the original of this post has been consigned to oblivion. I debated writing it

Last time we created a web service to classify ingredient quantities. This time we’ll use our web service along with the Food Data Central API to aggregate nutrient values for a recipe in a web app. Clean-up Before we start, we’ll do

Last time we finished getting our ingredient data ship-shape. We now have a function that does a pretty good job of mapping a rag-bag of free text ingredients to specific items in our database and calculating their aggregate nutritional values. Today we’ll look

Last time we improved our ingredient data. This time we’ll carry on with that by calculating more accurate density values. Food data central gives the mass in grams for particular portions of food in the file food_portion.csv. Where the portion is

Last time we generated a list of ingredients with mappings to items in the Food Data Central (FDC) database and very rough estimates for unit size and density. This time we’ll plug that data into the ingredient classifier we built previously and generate

Last time we successfully processed a recipe consisting of a list of free(ish)-text ingredient quantities, linked to items in a food database and calculated the number of grams of each macro-nutrient in the recipe. This was very effective but to accept a

Last time we built a classifier that reliably identified known ingredients and linked to their nutritional values in the Food Data Central database (https://fdc.nal.usda.gov/). This time we’re going to try and aggregate those values over all the ingredients in

Last time we looked at using our regex classifier to infer the part of the input string containing the ingredient using a set of rules. This meant we could identify the ingredients with only a little input knowledge but due to insufficient identifying information

Last time we built a classifier that uses a monte-carlo tree search algorithm to find a sequence of rules that can be applied to an input string to produce an output string with the desired classifications. This time we’ll make a couple of

Last time we used our regex classifier to start classifying our ingredient quantities. Unfortunately we were hampered by the need to join rules together as the only way of enforcing an ‘or’ relationship. It would be nice if we could just provide the list

Last time we built a simple text classifier from regexes. Today we’ll go back to our ingredient quantities example and see how far this new classifier can carry us towards extracting a canonical ingredient name and quantity from a relatively free-form input string.

Last time we noodled on the possibility of using regex substitutions to classify a string by applying type mark-up. Today, let’s implement a simple classifier in Python using this approach. Python regex substitutions The re module provides a method for substituting non-overlapping matches

Last time we looked at how to remove superfluous information from our input strings and we came up against a problem with the string: Olive oil (2 tbsp) In this instance, the (2 tbsp) would be removed and we would lose our amount information.

Last time we considered the causes of multiple representations of an item and how we might resolve them to produce a canonical form. As a quick reminder, the causes were: inconsistent casing or punctuation synonyms or local differences superfluous information varying specificity ordering Today,

Last time we implemented a lazy matcher, which gave us enough tools to separate ingredient name from amount (at least for our test dataset). This is great because now we can do things like convert the amounts to standard units such as grams or

Last time we did a bit of boring admin to tidy up the code and package it. So let’s do something more interesting today and look at lazy matching. To recap, we wanted a lazy matcher a few articles back to solve the

Over the past few articles we’ve been writing and testing an algorithm to comprehend text content. Before we make any further improvements, we’ll revisit the API design and formalise it into a library stored in a github repo so we can better

Last time we tested our classifier with 100 randomly picked ingredient quantity strings and discovered eight additional pieces of knowledge were required for accurate classification. A few avid readers correctly observed that I missed a couple of issues last time, so in fact we

Recap In the previous article we built a classifier that would take an input string along with some knowledge about its structure and produce an output string marked up with classifications. This worked nicely for the single case in hand but the question is

Following on from the previous article, we’re going to look at implementing our text classifier in Python. Tokenise Let’s start with the input string we considered last time: 6 - 7 cups of three different types of vegetables, chopped into bite size

The problem Suppose we have a string 3 tbsp tomato ketchup and we want to extract: ingredient name unit quantity It shouldn’t be too difficult to split this into the individual parts, right? Regex to the rescue? If we know that the string