Help Center
Development Technology Stack
Data Layer
Task | Technologies Used | Description | Tool | POC |
---|---|---|---|---|
Data Curation | Diffbot | Diffbot is used in to extract the required blog post(article) data elements | Data collection for Blogtrackers | Shaik |
Web Content Extractor | This tool can be used to collect both data elements and web URLs | Data collection for Blogtrackers | Shaik | |
Scrapy Regex crawler | For now, we use this too to collect Web URLs based on Regular expressions | Data collection for Blogtrackers | Shaik | |
YouTube Data API | Used to collect channels, videos, related videos, comments, and user engagement statistics from YouTube | VT | ||
YouTube DL | Collect videos for barcode generation | VT | ||
Data Cleaning | asyncio, aiohttp, aiomysql, pymysql, tqdm, requests, textblob, detoxify, pyautogui | Produces sentiment & toxicity values | BT, VT – backends | Shake |
Data Storage | MySQL | Storing and retrieving curated data | BT, VT | |
Data Manipulation | asyncio, aiohttp, aiomysql, pymysql, tqdm, requests, textblob, detoxify, pyautogui | Data Cleaning | BT, VT – backends |
Application Development Layer
Task | Technologies Used | Description | Tool | POC |
---|---|---|---|---|
Front End Design | Adobe XD, Affinity Designer, Invision Free Hand | Hayder | ||
Front End Development | VS Code, Google Chrome Developer Tools, Live Server, JS6, CSS3, HTML5, Github, Github Pages | BT, VT, FSA | Thomas | |
Front End | D3.js, ApexCharts | Used for graphs on the tools | BT, VT | Thomas |
Back End | Django Springboot | VT BT | Thomas Abiola | |
Data Access | MySQL, MariaDB, Elastic Search | VT, BT | Thomas, Abiola |
Deployment
Task | Technologies Used | Description | Tool | POC |
---|---|---|---|---|
Servers | Apache Tomcat, nginx | BT, VT | Thomas |
Testing
Task | Technologies Used | Description | Tool | POC |
---|---|---|---|---|
User Testing | Browsers – Firefox, Chrome, Edge, Safari | Test web applications in each browser for issues and bugs. Web application usability testing. | BT, VT |
Bug reporting
Task | Technologies Used | Description | Tool | POC |
---|---|---|---|---|
Bugs and Issues | ClickUp | Add and report issues and bugs to appropriate board in ClickUp | BT, VT |
Maintenance / Versioning
Task | Technologies Used | Description | Tool | POC |
---|---|---|---|---|
Version Control | Github | Code repository/version control | BT, VT |