Rafael D.Data Engineer
Skills
Computer Engineering student specialized in AI, with a focused trajectory towards a career in Data Science. Continuously expands and deepens expertise within this domain, demonstrating a commitment to ongoing learning and development. Possesses robust teaching abilities and excels in engaging with diverse audiences, effectively communicating complex concepts to both novices and experts alike.
Tech Lead
9/1/2022 - Present
Served as Tech Lead for the architecture and governance squad, with a primary responsibility for defining and managing comprehensive data architecture processes. Developed expertise in designing scalable and efficient data systems. Utilized a variety of tools and frameworks, including but not limited to, Hadoop, Apache Spark, and Kafka for big data processing and streaming analytics. Proficient in implementing numerous database systems such as SQL, NoSQL, and cloud-native databases like AWS Redshift and Google BigQuery. Demonstrated adeptness in data modeling, ETL processes, and data pipeline development using tools like Apache NiFi and Talend. Promoted collaborative workflows through efficient use of agile methodologies and version control systems, primarily Git. Enhanced data governance and security protocols by implementing best practices and compliance measures, ensuring data integrity and accessibility across the organization.Senior Data Engineer
11/1/2021 - 9/1/2022
Responsible for orchestrating the architecture of the B2B team from end to end, leveraging advanced technical skills in cloud infrastructure, microservices, and containerization. Developed proficiency in various programming languages and frameworks including Java, Spring Boot, and JavaScript. Utilized tools such as Docker and Kubernetes for container management and deployment. Engaged in database management using both SQL and NoSQL databases, specifically PostgreSQL and MongoDB. Ensured seamless integration and continuous delivery through CI/CD pipelines, employing tools such as Jenkins and GitLab. Demonstrated substantial success in leading the team’s efforts in API development and maintenance, employing RESTful services and GraphQL. Contributed to team collaboration and project coordination using Jira and Confluence, fostering a robust Agile/Scrum development environment. Through these efforts, significantly improved system scalability, reliability, and performance.Data Engineer
8/1/2021 - 11/1/2021
Worked as a data engineer with the Norway team, orchestrating Databricks pipelines and notebooks for Statkraft Energy Corp. Developed extensive proficiency in Databricks, utilizing MLflow for tracking experiments and managing models. Showcased advanced skills in Python and SQL for data manipulation and pipeline creation. Implemented and optimized ETL processes using Azure Data Factory and Azure Databricks. Employed Spark for large-scale data processing, ensuring data accuracy and efficiency. Maintained collaborative version control using Git, provided thorough documentation for workflows, and performed peer code reviews to uphold the highest standards of code quality. Utilized Apache Airflow for scheduling and managing workflows, ensuring seamless integration with other data systems.Data Engineer
2/1/2021 - 8/1/2021
Contributed to a project focused on constructing a data lake for a mining enterprise. Developed proficiency in data architecture and engineering principles relevant to large-scale data environments. Utilized Apache Hadoop, Spark, and Hive to implement scalable and efficient data storage and processing solutions. Employed ETL tools such as Apache NiFi and Informatica for seamless data integration. Demonstrated expertise in managing and organizing large datasets, ensuring high data quality and integrity. Showcased skills in SQL and Python for data manipulation and analytics. Employed technologies like AWS S3 and Redshift for cloud-based data storage and retrieval. Ensured the security and compliance of the data lake following industry standards. Collaborated with cross-functional teams to gather requirements and deliver solutions that meet business objectives.DBA Internship
6/2/2018 - 12/2/2018
During the internship, significant progress was made in data management and database administration. Expertise in SQL and database design principles was developed, focusing on tasks that ensured the integrity and accuracy of the company's database. Proficiency in using database management systems such as MySQL and PostgreSQL was achieved, with extensive work on data entry, data validation, and data cleanup processes. Skills in query optimization and performance tuning were honed, ensuring efficient retrieval and storage of data. Familiarity with version control systems like Git was gained to maintain and track changes in database schemas and codebase. The role involved close collaboration with the data analysis team to support their need for timely and accurate data, contributing to informed decision-making processes within the organization.
Computer Engineering at Federal University of Technology - Paraná
2015 - 2022Machine learning at SENAI (National Service for Industrial Training)
2012 - 2013
Microsoft Certified: Azure Data Engineer Associate at Microsoft
8/1/2022Advanced Excel at Masterfor
12/2/2020Power BI at Hashtag Treinamentos
12/2/2020Machine Learning A-Z: Hands On at SuperDataScience
12/2/2020
Rafael is available for hire
Meet Rafael D.
All Howdy Candidates are vetted for skills and english proficiency.
