Featured Projects

Sensor Based Activity Recognition

A deep learning challenge from my Data Science studies at FHNW

Goal:

The objective of this project was to create a system capable of detecting various activities passively monitored by a smartphone carried in a subject's pocket. The activities considered include Walking, Running, Stair Climbing, Cycling, Standing, and Sitting.

Approach:

Data collection for the six aforementioned activities was conducted following a specific protocol.
Preprocessing of data involved the removal of segments not covered by the protocol.
Development of an experimental pipeline facilitated the application of various preprocessing steps before training models.
Training was conducted on both a deep learning model and a non-deep learning model.
Evaluation of the model's performance was carried out.
The best performing model was made accessible via an API for deployment.
An app was developed to collect data and use the API for making predictions.

Conclusion:

We successfully developed and deployed a model capable of making predictions based on recorded data. Interestingly, the non-deep learning model outperformed the deep learning model, indicating that deep learning is not always the superior solution. The experimental pipeline enabled rapid and reproducible experimentation. However, the model is likely overfitted to the subjects due to the small sample size.

For more details consider visiting the project page:

Chatbot

A Retrievel Agumented Generation Task from my Data Science studies at FHNW

Goal:

The aim of this project was to develop a chatbot featuring a Retrieval-Augmented Generation (RAG) component alongside a conversation component that responds effectively to user concerns.

Approach:

Train a Large Language Model (LLM) to classify user prompts, deciding whether to use the RAG or the concern-specific LLM.
The RAG component includes a vector database holding chunks for the RAG system. Each chunk is assigned an embedding vector created by a sentence transformer, which is then utilized to retrieve the most appropriate chunks for answering questions. Subsequently, a causal LLM, fine-tuned for Question Answering, generates answers with the correct sources attached.
The concern model employed a pre-trained LLM, which was aligned through prompt engineering.
A Gradio dashboard was developed as the interface for bot interaction.

Conclusion:

We successfully fine-tuned and deployed the bot on a single NVIDIA GeForce RTX 4090 GPU. The path addressing user concerns responded effectively. While the fine-tuning of the RAG LLM was operational, it encountered difficulties when questions could not be answered with the provided context. Future work should include a user study to evaluate the bot's performance in real-life scenarios.

For more details consider visiting the project page:

Smart Classroom

Goal

The goal of this project was to develop a system to measure various metrics in a classroom using IoT sensors. The collected data should be uploaded to a database for storage.

Approach

Microcontrollers equipped with several sensors were used to measure temperature, humidity, CO2 levels, and the number of open windows. These microcontrollers run on CircuitPython and are powered by LiPo batteries, eliminating the need for long cables. Three measurement units (microcontrollers with sensors) were distributed throughout the classroom.

The data from the microcontrollers were transmitted to a Raspberry Pi via Bluetooth Low Energy (BLE), chosen for its low energy consumption. The Raspberry Pi, running a Python script, then sent the data to a server using SSL sockets via connected WiFi. The server then stored the data in a relational database.

To ensure continuous data collection, the server was programmed to send an email alert to a responsible person if no data was received for a certain period. This allowed for quick reactions if any issues occurred during data collection.

After the data was collected, it was analyzed to provide feedback to the teacher regarding the air quality in the classroom.

Conclusion

We successfully developed a system that collected and transmitted data from the microcontrollers to the server and then to the database. However, the main difficulty encountered was the instability of the BLE connection, which was prone to random interruptions. The key takeaway is that wireless connections, especially BLE, are not always stable and require careful management.