If you need support in completing the application or if you require a different format of this document, please get in touch with at UKI.Recruitment@tcs.com with the subject line: “Application Support Request”.Role: Pyspark Data EngineerJob Type: PermanentLocation: TCS GDC, Letterkenny, Co.Donegal, Ireland – Hybrid (3 days in office pw)- Ready to utilise your skills as a Pyspark Data Engineer, supporting major Banking Group Client within the EU?
- Are you looking for an exciting opportunity to contribute towards project/service delivery, grow professionally and work alongside a magnificent team?
- Are you keen to build your career with a dynamic global IT Consultancy and Customer focused team?
Join us as a Pyspark Data Engineer:Careers at TCS: It means moreTCS is a purpose-led transformation company, built on belief. We do not just help businesses to transform through technology. We support them in making a meaningful difference to the people and communities they serve - our clients include some of the biggest brands in the UK and worldwide. For you, it means more to make an impact that matters, through challenging projects which demand ambitious innovation and thought leadership.- Grow your career, while being exposed to new technologies.
- Be part of an exciting team where you will be challenged every day.
- Work with customers and identify opportunities to support their strategy and improve their processes across functions.
The RoleLooking for a Pyspark data engineer who is well versed with designing and optimizing data pipelines using Pyspark, processing large datasets, integrating data from various sources, ensuring data quality, and documenting workflows. Required skills include proficiency in Python, Pyspark, SQL, familiarity with big data tools like Hadoop and Hive, experience with cloud platforms such as GCP, and strong communication and teamwork abilities, with a preference for candidates having over three years of relevant experience.Key responsibilities:- Develop Data Pipelines: Design, implement, and data pipelines using Pyspark to process big data efficiently.
- Data Processing: Work with large datasets from various sources (structured, semi-structured, and unstructured) and transform them into usable formats for analytics and reporting.
- Optimize Spark Jobs: Write efficient Pyspark code, tune jobs for performance, and manage Spark clusters for reliability and speed.
- Data Integration: Integrate data from multiple sources, including databases, APIs, cloud storage.
- Data Quality & Validation: Implement checks for data accuracy, consistency, and completeness.
- Documentation: Prepare technical documentation for data pipelines, processes, and best practices to ensure knowledge sharing and maintainability.
- Support & Troubleshooting: Monitor data workflows, troubleshoot issues, and provide timely resolutions to maintain data system reliability.
Your ProfileKey skills/knowledge/experience: - Sound experience with Pyspark and big data technologies.
- Proficiency in Python: Extensive hands-on experience with Python, especially with data processing libraries and frameworks.
- Expertise in Pyspark: In-depth understanding of Spark’s core concepts (RDDs, DataFrames, Spark SQL) and ability to write efficient Pyspark code for distributed data processing.
- Big Data Ecosystem: Familiarity with Hadoop, Hive, HDFS, and related tools is often required
- SQL Skills: Strong SQL skills for querying, transforming, and aggregating data within Spark and other systems.
- Strong programming skills in Python and SQL.
- Solid understanding of ETL processes and data warehousing concepts.
- Excellent communication and teamwork skills.
Good to have:- Experience with cloud platforms (GCP) preferred.
Rewards & Benefits TCS is consistently voted a Top Employer in the UK and globally. Our competitive salary packages feature pension, health care, life assurance, laptop, access to extensive training resources and discounts within the larger Tata network.We offer health & wellness initiatives and sports events; we are the proud sponsor of the London Marathon.Diversity, Inclusion and Wellbeing Tata Consultancy Services UK&I is committed to meeting the accessibility needs of all individuals in accordance with the UK Equality Act 2010 and the UK Human Rights Act 1998.We welcome and embrace diversity in race, nationality, ethnicity, disability, neurodiversity, gender identity, age, physical ability, gender reassignment, sexual orientation. We are a disability inclusive employer and encourage disabled people to apply for this role. As a Disability Confident Employer, we offer an interview to applicants with disabilities or long-term conditions who meet the minimum criteria for the role. Please email us at UKI.recruitment@tcs.com if you would like to opt in.If you are an applicant who needs any adjustments to the application process or interview, please contact us at UKI.recruitment@tcs.com with the subject line: “Adjustment Request” email your recruiter at Rebecca.mcclelland@tcs.com . We welcome requests prior to you completing the application and at any stage of the recruitment process. Beware of Fraudulent offersThis is to notify you that TCS does not ask for any sort of payment or security deposit from candidates at any stage of the recruitment process. The firm never sends out job offers from free internet email services like Gmail, Yahoo Mail, and so on. TCS has not authorised any third-party company to collect money on their behalf. As a vigilant job seeker, beware of fraudulent recruitment activity and protect your interests! You can write to UKI.recruitment@tcs.com to report any fraudulent activity.Due to the high volume of applications, we will be unable to contact each applicant individually on the status of their application. If you have not received a direct response within 30 days, then it should be deemed unsuccessful on this occasion.Join us and do more of what matters. Apply online now.