- Designed, developed, and optimized ETL pipelines to extract, transform, and load raw data from diverse sources into business-level aggregates, supporting a large-scale data ecosystem with 2,900 tables and 141 TB of data. (Python, PySpark, Azure Databricks, Kafka, AWS S3, Google BigQuery, MySQL, Snowflake, YAML) - Engineered PII data processing services compliant with CCPA, GDPR, and CPRA, ensuring data security and privacy for 1,000 – 100,000 daily requests. (Python, PySpark, Azure Databricks, MySQL, Airflow) - Developed custom Apache Airflow operators to automate the migration of ETL workflows from Rundeck to Airflow, significantly reducing manual efforts and accelerating data pipeline deployment. (Python, Airflow, YAML, CI/CD) - Created cost-optimization pipelines and dynamic dashboards to identify and monitor cost-saving opportunities, reducing platform operational expenses by 12%. (Python, PySpark, Azure Databricks, MySQL, Data Visualization) - Built a scalable data ingestion framework leveraging 3rd-party APIs, ensuring robust data availability and expanding platform capabilities for new business requirements. (Python, REST APIs, Airflow, Data Orchestration) - Implemented CI/CD pipelines using GitLab CI/CD for automated deployment, enhancing service stability and secure infrastructure management. (Python, GitLab CI/CD, DevOps)

- Designed and implemented scalable streaming and batch ETL pipelines for data extraction from public sources, enabling daily ingestion of tens of gigabytes of raw data for analytical processing. (Python, PySpark, NoSQL) - Developed high-performance, parallel data parsers to bypass server restrictions, boosting data retrieval speed by 10x. (Python, Airflow, Web Scraping, REST API) - Applied machine learning algorithms and neural networks for NLP tasks, including text classification and clustering, improving data quality for advanced analytics. (Python, PySpark, Scikit-learn, Keras, NLP, Data Science) - Engineered back-end APIs and data services for a web platform to deliver business-ready data to stakeholders. (Python, Django, JavaScript, NoSQL, REST APIs)

- Provided comprehensive support and actively participated in the development of the company's website for the interaction of clients and tender agents, which allowed hundreds of clients to comfortably interact with agents daily (Python, Django, ORM, DRF, VueJS, MySQL) - Developed components for extracting data on the availability of tenders from open sources and uploading it to CSV format, which reduced data processing time from 3 hours of manual work to 15 minutes of parser work. (Python, Django, VueJS)