Projects

I - Description

Situation: Come up with a startup idea of an innovative service based on:

• IOT devices
• An information system

IOT Devices: These devices must emit data every few minutes (or seconds). Assuming the startup is successful, there will be millions of these devices. We consider that you've managed to make a working prototype. Every data emitted by the devices must contain:

• Timestamps
• Device ID
• Latitude and Longitude
• Any other field you find useful (e.g., temperature, humidity, etc.)

Your Information System Must Provide:

• An urgent service referred to as "alert"
• A long-term analytics service

Recommended System Architecture: Based on the preliminary questions, your solution is very likely to include:

• At least one distributed storage
• At least one distributed stream processing framework
• At least two stream consumers

II - Our solution

Our idea was to assess the danger posed by individuals and alert the police when necessary. We use a "dangerousity" index, a float ranging from 0 to 1. For testing the architecture, we simply generated a random number within this range.

We implemented a Kafka stream with two brokers, each configured with 3 partitions and replicates across all topics. This setup ensured that even if a broker failed, the project would continue functioning without any data loss, demonstrating our project's scalability.

There were three sources of data, each representing drones, transmitting data every 0.1 seconds. All data were sent to a Kafka stream under the topic "drone-data."

A consumer named "Consumer Datalake" stores the data on the Minio datalake. Another consumer, "Consumer Alert," monitors the "dangerousity" index. If it exceeds 0.9, it forwards the data to the topic "high-danger-alerts."

Additionally, "Consumer AlertProcess" is tasked with triggering alerts via the Discord API to a Discord server when high danger alerts are received.

Below is the architectural diagram that summarizes our setup:

I - Description

Competition 1

• Write a CNN by hand with fewer than 30 layers (include print("Number of layers: ", len(model.layers)) in your sheet; I will grep it).
• Submit your results.
• Check your ranking (and possibly start over).

Competition 2

• This involves starting with an existing network and adapting it for this project. You are allowed to do anything, but it must be able to run on Kaggle.
• Submit your results.
• Check your ranking (and possibly start over).

II - Hand made competiton

I tried copying different existing architecture like AlexNet or VGGNet, I also tried to build my own but it was not very concluent all this was done with keras Tensorflow.

In the end I took the VGGNet and did some testing around it which alowed me to have a testing accuracy of 0.91141 on images we didn't had access to. Alowing me to be in the 4th place of the competition.

III - Fine Tuning competition

We tried fine tuning different models and the one that worked the best for this exercise was a version of pre trained densenet121 from Pyrtorch.

And with testing we went up to 0.94891 on the images we didn't had access to. Alowing me to be in the 1st place of the competition.

I - Description

5 years of financial data

The work is done in two steps:

• Read the data, clean it, and store it intelligently in a database.
• Create a dashboard that allows for visualization of the data and performing some analyses.

II - Backend

The backend was the most substantial component of this project. It involved collecting, processing, and optimizing the data with Pandas, plus transferring it to the PostgreSQL database, which was challenging.

After optimizing the code, the total processing time for all the data was approximately one hour to read the datas from 271325 files, clean them and stock the datas, even on less efficient computers.

One potential improvement to reduce processing time could be to implement multithreading. However, I encountered database issues and lacked sufficient time to address these. Additionally, the project's high resource consumption with multithreading limited its compatibility across different computers.

III - Frontend

In our project, we've created a financial analytics dashboard using Dash in Python. The features we did were:

• Display options for stock prices on a logarithmic scale, either as line charts or candlesticks.
• Customizable date range selections for viewing stock data.
• Multiple stock displays with individual color coding and an interactive legend for toggling visibility.
• Bollinger Bands for analyzing stock volatility.
• A data table showing daily stats like min, max, and average prices for selected stocks.

I - Description

"AI vs AI": concerns the detection of texts emitted by a generative AI. This challenge takes place at Albertschool in Paris during three days.

II - Data cleaning and Pretreatment

During the data cleaning phase, we performed basic operations such as removing datas that were either NULL or not usable.

Subsequently, we applied various tokenization methods and assessed their performance across different models, utilizing libraries such as NLTK.

We also reviewed all sources of the texts and identified that the dataset was constructed using more than 30 different AIs.
With this knowledge, we adjusted the number of texts sourced from each AI based on their initial accuracy in our existing model.

III - Models

We experimented with a variety of models for this project, compared their performances, and selected the best one. Here is the list of the different models we tested:

TF-IDF with GridSearch	F1-score: 0.9516
N-BAGS with GridSearch	F1-score: 0.9507
BiDirectionnal RNN	F1-score: 0.9437
Bert	F1-score: 0.8137

I - Description

1- Data Processing:

• Presentation: Describe the chosen dataset.
• Pre-processing: Apply tokenization (regex and BPE), normalization (removal of stop words, lemmatization, converting to lowercase).
• Statistics: Provide basic statistics (number of documents, tokens, frequency of tokens, etc.).

2- Models:

• Training: Train various models (n-gram, naive Bayes, logistic regression, tf-idf, word2vec, neural networks).
• Evaluation: Assess performance using standard metrics (perplexity, recall, precision, f1-score).
• Optimization: Vary pre-processing and hyperparameters to improve performance.
• Analysis: Discuss limitations and propose improvements.

II - Data cleaning and Pretreatment

We analyzed the dataset and then cleaned the data to ensure all entries were in English, eliminating any multilingual issues for training our models.

We implemented different Tokenization processes using NLTK and Tiktoken, which segment the text into smaller units. We also applied several normalization methods to simplify the text, such as converting to lowercase, removing punctuation, stopwords, and lemmatization.

III - Models

The project explores several models for the classification of music genres.
N-grams use trigrams to generate new content, showing potential in capturing the lyrical styles of musical genres.
Naive Bayes and Logistic Regression are tested with different Preprocessing to examine their effectiveness in classification. Advanced approaches such as TF-IDF and Word2Vec are also implemented to enhance the capture of semantic relationships in the texts.
Neural Networks, both feedforward and recurrent, are experimented with to assess their ability to handle the sequential data of lyrics.
Each model is evaluated by its Accuracy, Recall, and F1-score to determine which method provides the best results for genre classification.

I - Description

Utilize datasets of your choice to implement Computer Vision and Machine Learning using Microsoft Azure.
To build a website for the cloud-based Machine Learning component, you must have access to an API provided by Microsoft Azure.
For the Computer Vision component, you need to construct a Docker container to access its API.

II - Our solution

We used Microsoft Azure tools to train a Computer Vision model, which we then deployed locally using Docker Docker.
Then, we employed Machine Learning capabilities to analyze tabular data.
These combined technologies enabled us to identify various stages of cancer effectively with different data types.
To integrate these components, we developed a streamlined Flask application.

Below is the architectural diagram that summarizes our setup:

This Website is meticulously designed to showcase the complete Portfolio of projects I have created.
I aimed to achieve a Clean and Refined aesthetic throughout the design, ensuring that each project is highlighted in a visually appealing and organized manner.

There is also a small GeoGuessr like game hidden in the Website.
I used the Google API and JavaScript to do it, but there are only two mounth of free API usage so it wont be able for a long time.

TO DO...