About this Project

For our industrial project with ARM, we were tasked to build a speech recognition model and run it on a highly resource constrained micro controller.

To accomplish this task we had to collect training data, design a good but constrained neural network, and then train the neural network on the training data.

Once a network was trained, it was to be quantized, i.e. turned into a fixed-point representation, such that it can run in an adequate time and consume less memory on the micro controller.

Finally, we had to built and test a control code for the micro controller, which is a C program that includes the Tensorflow lite library, as well as facilities to load and run the model obtained in the previous step.

This program is also responsible for gathering input data from the sensors, as well as communicating the inference results back to the outside world.

Our final product packaged all of these pieces up into a GUI written in Python.

My Contribution

• I leveraged grid search algorithms to determine good small model candidates and train them on our available data
• I wrote C routines to decode and load the model into Tensorflow Lite, as well as run the model on the micro controller and communicate with the sensors
• Collaborated intensively with my group members to integrate these parts of the process with a unified GUI written in Python

The micro controller we used to run the neural network.

The Project

The goal of the project was to compute the expression:

0.5x + x^2 * cos((x - 128) / 128)

The initial task was to implement this function in C and use it to compute this formula over a large range of inputs x, and then compute the sum of all the intermediate results.

For the second task, I designed a custom instruction for 0.5 * a + b, with floating point inputs a and b, which can be used for part of the above formula.

The key step to computing the above formula efficiently is to compute the cosine of some value. To do so efficiently, I implemented the CORDIC algorithm as a custom instruction, operating on fixed point binary inputs.

CORDIC will compute the sin and cosine simultaneously, but for the above formula only the cosine was required and used.

In the final part of the project, these parts where combined into a single instruction. Once this was accomplished, I further improved upon the resource and memory utilization by implementing unrolled and partially-unrolled pipe-lined designs.

All these circuits where designed using the Verilog HDL, and tested using simulations, as well as by running on actual FPGA hardware.

The Project

Using an embedded processor and a keyboard input device, we built a small electric piano. The piano can play multiple keys at once, plays correct wave-forms for the musical notes by generating overlapping sinus wave-forms in real time as keys are pressed.

It also modulates the amplitude of each note component, depending on how long the corresponding key has been pressed for, reducing the amplitude (loudness) over time to simulate a physical piano.

My Contributions

I programmed the display logic and interrupts, which are responsible for decoding the inputs coming from the key-pad. The display shows which chord is pressed in real time.

My other major contribution was to propose and implement the use of the CORDIC algorithm to efficiently generate the sign wave-forms. On the resource-constrained micro controller, computing these is a non-trivial task.

To enable as many keys as possible to be pressed, but also retain a good quality audio, the CORDIC algorithm can adapt to take varying numbers of cycles depending on how many keys are pressed simultaneously. This allows for more keys to be pressed, while preserving the resolution of the wave-form when only one or two keys are active.

About the project

For this project we conceptualized, designed, and implemented an embedded device to monitor plants at home. This works by tracking the water level of plants over time (taking into account the humidity and other environmental factors), to calculate when the plant should next be watered.

To notify the users of the need to water, we built a physical gauge that can be placed next to the plant, a button to indicate that it has been watered, as well as a web service to check on your plants on the go.

What I did

I implemented communication with different sensors to measure humidity and Co2 concentration.

This was achieved by studying the sensors datasheets and then implementing their protocols in Python to address the sensors which were connected via the GPIO headers of a raspberry pi.

In a similar way, I also implemented controls for registering the button click, and controlling a stepper motor to move the gauge.

To display the water level, I built a physical gauge and connected it to the motor.

Finally to build the web interface I set up a python based web server. This server responds to HTTP requests, and receives updates from our device via the MQTT protocol. I also implemented the MQTT protocol on the device side to send updates, and set up a local MQTT broker to allow for real time communication between both parts.

About the Project

Translator

In the first deliverable we built a parser for a subset of the C programming language and a C++ compiler which can be used to generate Python code.

The compiler/ translator for this part supports the following constructs:

Variables:

• Local integer variables (though not nested scopes)
• Global integer variables
• No variable shadowing

Functions:

• Definition of functions taking 0 or more integer parameters and with void or integer return type
• Explicit support for main special functionality

Statements:

• if
• if-else
• while
• return
• sequences

Expressions:

• Decimal integer constants
• Integer arithmetic: *,+,-
• Logical operations: &&,||
• Comparison operations: <,==
• Invocation of functions
• Assignment (but only as a direct expression statement)

MIPS Compiler

For this part of the project, we used the parser and C++ logic we had in place from the translator, and extended it to build a compiler which generates MIPS assembly code.

We started by implementing a simpler feature-set initially, to be able to compile:

• a file containing just a single function with no arguments
• variables of int type
• local variables
• arithmetic and logical expressions
• if-then-else statements
• while loops

Once those constructs where working, we moved on to handling progressively more and more complex programs which include:

• files containing multiple functions that call each other
• functions that take up to four parameters
• for loops
• arrays declared globally (i.e. outside of any function in your file)
• arrays declared locally (i.e. inside a function)
• reading and writing elements of an array
• recursive function calls
• the enum keyword
• switch statements
• the break and continue keywords