Summary
The Wikimedia Foundation is seeking an experienced engineering leader to serve as Director of Data Engineering for our Technology department. At the Wikimedia Foundation, we operate the world’s largest collaborative project: a top ten website, reaching a billion people globally every month, while incorporating the values of privacy, transparency and community that are so important to our users.
Reporting to the VP of Data Science &Engineering, the Director of Data Engineering is a key member of the Foundation’s data team and an active participant in the strategic decision making framing the work of the technology department, the Wikimedia Foundation and the Wikimedia movement. Working closely with other Technology and Product teams, as well as our community of contributors and readers, you will help shape the next generation of data usage, analysis and access across all Wikimedia projects.
This role is responsible for planning and executing on key data engineering initiatives spanning our work in artificial intelligence, machine learning, search, natural language processing and product analytics. As Director, you will lead a team of data engineers, site reliability engineers and software engineers in building and maintaining our entire analytics and events infrastructure, including key systems like Kafka, Cassandra, Druid and our Hadoop/Spark cluster. We build our stacks using fully open source components, we release our data publicly and our puppet repo is public.
Here are some examples of projects we have tackled that you may be excited to lead:
- Releasing public data to the wikimedia community and the world at large. Our public data offerings are used all over the world by companies and research institutions big and small. A popular example is the Wikipedia Clickstream (a.k.a. Wikipedia's rabbit hole)
- Deploying an anomaly detection system to monitor Wikipedia accessibility over the world and detect possible outages (or censorship events)
- Integrating an open-source GPU software platform like AMD ROCm in Hadoop and in the Tensorflow-related ecosystem
- Building the Foundation's new event data platform infrastructure
- Migrate our on-prem Hadoop infrastructure that holds petabytes of data to Hadoop3
- Evangelize privacy conscious ways to compute metrics. Privacy is key to the work we do
We’d like you to do these things:
- Partner closely with other teams and departments across the Wikimedia Foundation to define and experiment with machine learning products. These could be brand new feature offerings in Wikipedia or augmentation of existing workflows.
- Review and advice in code changes and technical decisions made by team
- Represent team members within the organization and Wikimedia community
- Support and coach your team members in the development of their career paths
- Recruit and hire new team members
- Work closely with our Research, Architecture, Security, Site Reliability and Platform teams to define our next generation of data architecture
- Ensure data is available, reliable, consistent, accessible, secure, and available in a timely manner for external and internal stakeholders and in accordance with our privacy policy
- Contribute to our culture by managing, coaching and developing team members
Skills and Experience:
- Deep experience in leading data science, machine learning, search or data engineering teams that is able to separate the hype in the artificial intelligence space from the reality of delivering production ready data systems
- 5+ years engineering leadership experience
- Demonstrated ability to balance competing interests in a complex technical and social environment
- Proven success at all stages of the engineering process and product lifecycle, leading to significant, measurable impact
- Previous hands-on experience in production big data and machine learning environments at scale. You are no stranger to error budgets, operations at scale and efforts to reduce toil.
- Experience building and leading diverse, international and remote-first teams
Qualities that are important to us:
- You take a solutions-focused approach to challenging data and technical problems
- A passion for people development, team culture and the management of ideas
- You have a desire to show the world how data can be done while honoring the user’s right to privacy
Additionally, we’d love it if you have:
- Experience with modern machine learning, search and natural language processing platforms
- A track record of open source participation
- Fluency or familiarity with languages in addition to English
- Spent time having lived or worked outside your country of origin
- Experience as a member of a volunteer community
The Wikimedia Foundation is...
...the nonprofit organization that hosts and operates Wikipedia and the other Wikimedia free knowledge projects. Our vision is a world in which every single human can freely share in the sum of all knowledge. We believe that everyone has the potential to contribute something to our shared knowledge, and that everyone should be able to access that knowledge, free of interference. We host the Wikimedia projects, build software experiences for reading, contributing, and sharing Wikimedia content, support the volunteer communities and partners who make Wikimedia possible, and advocate for policies that enable Wikimedia and free knowledge to thrive. The Wikimedia Foundation is a charitable, not-for-profit organization that relies on donations. We receive financial support from millions of individuals around the world, with an average donation of about $15. We also receive donations through institutional grants and gifts. The Wikimedia Foundation is a United States 501(c)(3) tax-exempt organization with offices in San Francisco, California, USA.
As an equal opportunity employer, the Wikimedia Foundation values having a diverse workforce and continuously strives to maintain an inclusive and equitable workplace. We encourage people with a diverse range of backgrounds to apply. We do not discriminate against any person based upon their race, traits historically associated with race, religion, color, national origin, sex, pregnancy or related medical conditions, parental status, sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, or any other legally protected characteristics.
If you are a qualified applicant requiring assistance or an accommodation to complete any step of the application process due to a disability, you may contact us at recruiting@wikimedia.org or (415) 839-6885.
U.S. Benefits &Perks*
- Fully paid medical, dental and vision coverage for employees and their eligible families (yes, fully paid premiums!)
- The Wellness Program provides reimbursement for mind, body and soul activities such as fitness memberships, baby sitting, continuing education and much more
- The 401(k) retirement plan offers matched contributions at 4% of annual salary
- Flexible and generous time off - vacation, sick and volunteer days, plus 22 paid holidays - including the last week of the year.
- Family friendly! 100% paid new parent leave for seven weeks plus an additional five weeks for pregnancy, flexible options to phase back in after leave, fully equipped lactation room.
- For those emergency moments - long and short term disability, life insurance (2x salary) and an employee assistance program
- Pre-tax savings plans for health care, child care, elder care, public transportation and parking expenses
- Telecommuting and flexible work schedules available
- Appropriate fuel for thinking and coding (aka, a pantry full of treats) and monthly massages to help staff relax
- Great colleagues - diverse staff and contractors speaking dozens of languages from around the world, fantastic intellectual discourse, mission-driven and intensely passionate people
*Please note that for remote roles located outside of the U.S., we defer to our PEO to ensure alignment with local labor laws.
More information
Wikimedia Foundation
Blog
Wikimedia 2030
Wikimedia Medium Term Plan
Our Commitment to Equity
This is Wikimedia Foundation
Facts Matter
Our Projects
Our Tech Stack