Role Profile
IDPS (Index Data Processing Platform) is a program whose main purpose is to drive improvements across data collection, data management and data governance resulting in improved efficiencies, robustness, and the ability for our business to use our data as an asset by driving both cost savings and revenue opportunities.
Some of the outcomes and benefits of this program include:
- Provides a single, centralized version of the truth/data including standardized logical and physical data model
- Data capabilities to build world class multi-asset indices and data & analytics services
- Universal data quality profiling and checks as well as Data Policies and Governance
- A platform for data discovery, management and cataloguing
- Ability to on-board and integrate new business acquisitions more efficiently
- Improve operational efficiency through automation
- A customized bi-directional packaging and delivery capability
Role Description:
- We are looking to fill a Python ETL Developer who will play an integral role in the ongoing application & data analysis, design, development, integration, enhancements and maintenance of the data platform which will span a broad range of data and functionality. While constantly driving innovation and working on complex technical challenges every day, this team member will work closely with BAs, PO’s, PMs and other team members to understand business requirements, design and develop quality technical solutions that are aligned with the business and platform strategies while ensuring compliance with the organization’s architectural and coding standards. In this role, the developer will also be responsible for helping to troubleshoot complex support issues on an as needed basis. This role will be based in Malabe – Sri Lanka, Bucharest – Romania on the MIT – Shared Tech Team.
Key Responsibilities:
- Work together with other developers, business analysts and product owners to ensure that new or modified ETL applications are developed using industry standard best practice design and development practices
- Contribute to the development, delivery and maintenance of ETL solutions
- Design, prototype and develop optimal code to maintain exceptional performance in processing large volumes of data across diverse sources
- Build solutions using SAP HANA and/or AWS Cloud native ecosystem and components
- Design, develop and test data models and ensure that data is correctly loaded into the system
- Application performance analysis and performance optimization
- Defining and maintaining robust and scalable ETL to manage data according to formal concepts, principles, and techniques
- Design and develop scalable, reusable ETL code in line with architectural standards and best practices
- Make recommendations for data engineering related functions that result in a more cost-effective product delivery
- Help to manage data as a critical asset within IDPS business functions
Work Experience:
- 3+ years hands on experience in Python based ETL Application development experience
- Experience in executing/monitoring ETL jobs/pipelines and data engineering support on production environment
- Sound knowledge & hands on experience in troubleshooting and performance optimization of Python ETL applications and SQL code.
- Hands on experience in developing ETL jobs, workflows & data-pipelines.
- Experience with Python frameworks such as Bonobo, pygrametl is a plus
- Experience in data transformation features such as: Preserving, Hierarchy Flattening, Match Transform, Data Cleansing Generator, Lookup, Fuzzy Lookup, etc.
- Hands on experience implementing SCD Type 2 dimensions, to maintain the history of dimensional data
- Experience in Data Warehousing or Data Lakes which includes Data integration concepts, analysis and development of Dataflows, mappings using needed transformations
- Experience with end-to-end large Data Conversion Projects to migrate large volumes of data from SQL DBs, NoSQL DBs, File sources, etc. to a target database/warehouse.
- Exposure on open-source big data stacks (preferred kudu/impala/hive/kafka/flink)
- Experience in Cloud Native Big Data stack & Data Warehousing implementations such as Snowflake is a plus.
- Experience with Financial Markets domain is a plus
- Experience in BI tools is a plus
- Experience using SAP ETL Tools such as Data Services and Data Intelligence is a plus
- SAP Experience with SLT, SDI a plus
- Knowledge of SAP HANA Advanced Modelling (SAP HANA Text, SAP HANA Graph, SAP HANA Spatial, SAP HANA PAL) a plus
- Experience Developing SAP HANA XS/XSA applications based on the HANA architecture a plus
Education:
- Bachelor’s Degree in MIS, Computer Science, or IT related field or equivalent IT related experience
IT Skills:
- Solid understanding of databases and SQL/Analytical SQLHands on experience working with the IDE’s like Zeppelin, Jupyter, Notebook, PyCharm etc
- Hands on experience in using build/deployment tools such as PyBuilder, Pip, Jenkins, Docker
- Hands on experience in using Pandas, PySpark, Kafka, Boto3, etc.
- Hands on experience on ETL orchestration tools – preferably SAP HANA, Nifi, Airflow, AWS Glue, AWS DMA
- Fundamental understanding of ETL stages and techniques
- Demonstrated knowledge & experience in OOP/OOD
- Demonstrated ability to understand business requirements and the implications of those requirements on current and future roadmaps
- Understanding, at a detailed level, of all architectural dependencies of technologies in IT environments
- Experience with scripting languages such as javascript is a plus
Soft Skills:
- Self-motivated and proactive with demonstrated critical thinking and creative problem-solving skills
- Agile development processes (SCRUM and Kanban)
- Strong communication, presentation, documentation and interpersonal skills
- Able to self-manage and work independently in a fast-paced environment with dynamic requirements and priorities
- Experience with Jira a plus
Diversity & Inclusion
People are at the heart of what we do and drive the success of our business. Our colleagues thrive personally and professionally through our shared values of Integrity, Partnership, Innovation and Excellence are at the core of our culture. We embrace diversity and actively seek to attract people with unique backgrounds and perspectives. We are always looking at ways to become more agile so we meet the needs of our teams and customers. We believe that an inclusive collaborative workplace is pivotal to our success and supports the potential and growth of all colleagues at LSEG.
London Stock Exchange Group | Closing Date: Apply Now