Help Center
Development Technology Stack
Data Layer
| Task | Technologies Used | Description | Tool | POC |
|---|---|---|---|---|
| Data Curation | Diffbot | Diffbot is used in to extract the required blog post(article) data elements | Data collection for Blogtrackers | Shaik |
| Web Content Extractor | This tool can be used to collect both data elements and web URLs | Data collection for Blogtrackers | Shaik | |
| Scrapy Regex crawler | For now, we use this too to collect Web URLs based on Regular expressions | Data collection for Blogtrackers | Shaik | |
| YouTube Data API | Used to collect channels, videos, related videos, comments, and user engagement statistics from YouTube | VT | ||
| YouTube DL | Collect videos for barcode generation | VT | ||
| Data Cleaning | asyncio, aiohttp, aiomysql, pymysql, tqdm, requests, textblob, detoxify, pyautogui | Produces sentiment & toxicity values | BT, VT – backends | Shake |
| Data Storage | MySQL | Storing and retrieving curated data | BT, VT | |
| Data Manipulation | asyncio, aiohttp, aiomysql, pymysql, tqdm, requests, textblob, detoxify, pyautogui | Data Cleaning | BT, VT – backends |
Application Development Layer
| Task | Technologies Used | Description | Tool | POC |
|---|---|---|---|---|
| Front End Design | Adobe XD, Affinity Designer, Invision Free Hand | Hayder | ||
| Front End Development | VS Code, Google Chrome Developer Tools, Live Server, JS6, CSS3, HTML5, Github, Github Pages | BT, VT, FSA | Thomas | |
| Front End | D3.js, ApexCharts | Used for graphs on the tools | BT, VT | Thomas |
| Back End | Django Springboot | VT BT | Thomas Abiola | |
| Data Access | MySQL, MariaDB, Elastic Search | VT, BT | Thomas, Abiola |
Deployment
| Task | Technologies Used | Description | Tool | POC |
|---|---|---|---|---|
| Servers | Apache Tomcat, nginx | BT, VT | Thomas |
Testing
| Task | Technologies Used | Description | Tool | POC |
|---|---|---|---|---|
| User Testing | Browsers – Firefox, Chrome, Edge, Safari | Test web applications in each browser for issues and bugs. Web application usability testing. | BT, VT |
Bug reporting
| Task | Technologies Used | Description | Tool | POC |
|---|---|---|---|---|
| Bugs and Issues | ClickUp | Add and report issues and bugs to appropriate board in ClickUp | BT, VT |
Maintenance / Versioning
| Task | Technologies Used | Description | Tool | POC |
|---|---|---|---|---|
| Version Control | Github | Code repository/version control | BT, VT |