Data Engineer

Capital One is seeking a highly skilled Lead Data Engineer to join our Finance Technology team. As a Lead Data Engineer, you will be responsible for designing, developing, and deploying large-scale data pipelines and frameworks using open-source tools on public cloud platforms.

• Collaborate with Agile teams to design, develop, test, implement, and support technical solutions in full-stack development tools and technologies.
• Leverage ETL programming skills in open-source languages, including Python, Scala, and SQL, on various frameworks.
• Provide technical guidance concerning business implications of application development projects.
• Deploy DevOps techniques and practices, such as Continuous Integration, Continuous Deployment, Test Automation, Build Automation, and Test-Driven Development, to enable rapid delivery of working code using tools like Jenkins, Nexus, Maven, GitHub, and Docker.
• Share your passion for staying on top of tech trends, experimenting with and learning new technologies, participating in internal and external technology communities, and mentoring other members of the engineering community.
• Perform unit tests and conduct reviews with other team members to ensure your code is rigorously designed, elegantly coded, and effectively tuned for performance.
• Manage multiple responsibilities in an unstructured environment where you are empowered to make a difference.

• Bachelor's Degree.
• At least 6 years of experience in application development (internship experience does not apply).
• At least 2 years of experience in big data technologies.
• At least 1 year of experience with cloud computing (AWS, Microsoft Azure, Google Cloud).

• 7+ years of experience in application development, including Python, SQL, Spark-Scala, or Java, PySpark on Maven builds, REST, JSON, relational databases, and CICD on Jenkins.
• 7+ years of experience developing ETL solutions.
• 4+ years of experience developing, deploying, testing in AWS public cloud.
• 4+ years of experience with cloud computing, preferably AWS and its services, including deploying S3, EMR/EC2, SNS, SQS, and Lambda functions.
• 4+ years of experience with distributed data/computing tools (MapReduce, EMR, S3, Lambda, Kafka, Spark, or MySQL).
• 4+ years of experience delivering large-scale dataset solutions and SDLC best practices.
• 4+ years of experience working on real-time data and streaming applications.
• 4+ years of experience with NoSQL implementation (Mongo, Cassandra).
• 4+ years of experience with UNIX/Linux, including basic commands and shell scripting.
• 2+ years of experience with Agile engineering practices.
• AWS Certification.

Capital One is an Equal Opportunity Employer committed to diversity and inclusion in the workplace. All qualified applicants will receive consideration for employment without regard to sex, race, color, age, national origin, religion, physical and mental disability, genetic information, marital status, sexual orientation, gender identity/assignment, citizenship, pregnancy or maternity, protected veteran status, or any other status prohibited by applicable national, federal, state, or local law.