Responsibilities
- Design, develop, and maintain scalable data pipelines and systems.
- Monitor and troubleshoot data pipeline issues to ensure seamless data flow.
- Establish data processes and automation based on business and technology requirements, leveraging Visa’s supported data platforms and tools
- Deliver small to large data engineering and Machine learning projects either individually or as part of a project team
- Setup ML Ops pipelines to Productionalize ML models and setting up Gen AI pipelines
- Collaborate with cross-functional teams to understand data requirements and ensure data quality, with a focus on implementing data validation and data quality checks at various stages of the pipeline
- Provide expertise in data warehousing, ETL, and data modeling to support data-driven decision making, with a strong understanding of best practices in data pipeline design and performance optimization
- Extract and manipulate large datasets using standard tools such as Hadoop (Hive), Spark, Python (pandas, NumPy), Presto, and SQL
- Develop data solutions using Agile principles
- Provide ongoing production support
- Communicate complex concepts in a clear and effective manner
- Stay up to date with the latest data engineering trends and technologies to ensure the company's data infrastructure is always state-of-the-art, with an understanding of best practices in cloud-based data engineering
This is a remote position. A remote position does not require job duties be performed within proximity of a Visa office location. Remote positions may be required to be present at a Visa office with scheduled notice.
Qualifications
Primary / Basic Skills
- 3 or more years of work experience with a Bachelor’s Degree or more than 2 years of work experience with an Advanced Degree (e.g. Masters, MBA, JD, MD)
- 3+ years of work experience with a bachelor’s degree in the STEM field.
- Strong experience with SQL, Python, Hadoop, Spark, Hive, Airflow and MPP data bases
- Experience with both traditional data warehousing tools and techniques (such as SSIS, ODI, and on-prem SQL Server, Oracle) as well as modern technologies (such as Hadoop, Denodo, Spark, Airflow, and Python)
- Advanced knowledge of SQL (e.g., understands subqueries, self-joining tables, stored procedures, can read an execution plan, SQL tuning, etc.)
- Solid understanding of best practices in data warehousing, ETL, data modeling, and data architecture.
- A team player and collaborator, able to work well with a diverse group of individuals in a matrixed environment
Preferred Skills
- A solid understanding of best practices in data engineering
- Strong analytics experience with a focus on Data Engineering and AI
- Experience with cloud-based data warehousing and data pipeline management (AWS, GCP, Azure)
- Experience with NoSQL databases (e.g., MongoDB, Cassandra)
- Maintainance and Support of Data pipelines (daily / monthly)
- Experience with visualization software (e.g., Tableau, QlikView, PowerBI) is a plus.