Senior Site Reliability Engineer (Distributed Storage) - remote

Summary

We are looking for a Senior Site Reliability Engineer to support and develop the platform serving the world’s favorite encyclopedia to millions of people around the globe. Wikipedia and its sister projects are powered strictly by Free and Open Source software with MediaWiki at its core surrounded by an ecosystem of microservices in PHP, NodeJS, Python, Go and Java.

The Wikimedia community generates a sizable amount of data, including freely-licensed media. As part of our team you would have the opportunity to solve storage, scaling, access and backup challenges Wikipedias’scale provides.

We currently use Openstack Swift as our main object-storage system and Cassandra as the dominant NoSQL solution. We are interested in finding new and improved ways of utilizing the capabilities of the existing solutions as well as researching and potentially implementing other similar products.

We are a globally distributed and diverse team of engineers with a drive to explore, experiment and embrace new technologies. If you find what we do interesting, if you are up to the challenge of improving the reliability and delivery of one of the Internet’s top 10 websites, and you enjoy the idea of working in a remote-first role, we may just be the right place for you.

Responsibilities

• Design, improve and operate the distributed storage and NoSQL systems serving Wikimedia projects
• Research, test and benchmark new storage and NoSQL solutions
• Respond to common database issues, following documented procedures
• Collaborate with SRE and other teams across the organization to ensure their specific storage, backup and DR needs are met on an ongoing basis
• Perform day-to-day operational/DevOps tasks on Wikimedia’s public facing infrastructure (deployment, maintenance, configuration, troubleshooting)
• Implement and utilize configuration management and deployment tools
• Assist in the architectural design of new services and making them operate at scale
• Monitor systems, services and service clusters, optimize performance and resource utilization
• Assist in or lead incident response, diagnosis and follow-up on system outages or alerts across Wikimedia’s production infrastructure
• Mentor peers in your areas of technical and operational strength
• Share our values and work in accordance with them

Qualifications

• 3+ years experience in an SRE/Operations/DevOps role as part of a team
• Experience with (Free and Open Source) distributed storage products (Openstack Swift, Ceph, etc.)
• Experience with NoSQL products (Cassandra, Redis, Elasticsearch, etc.)
• Comfortable with shell and scripting languages used in an SRE/Operations engineering context (Python, Go, Bash, Ruby, etc.)
• History of automating tasks and processes, identifying process gaps and finding automation opportunities
• Strong English language skills and ability to work independently, as an effective part of a globally distributed team
• B.S. or M.S. in Computer Science or equivalent work experience

Pluses

• Familiarity with database administration and support, debugging query performance and schema design
• Experience managing data backups and knowledge of popular Open Source backup management software (bacula, amanda, rdiff-backup/duplicity, etc…)
• Experience with centralized storage systems (SAN and NAS) and/or tape storage systems
• Low level systems troubleshooting and debugging (CPU/memory profiling, C/C++ experience, in-depth Linux knowledge)
• Experience with LAMP stack technologies (PHP/HHVM, memcached/redis, MySQL) - MediaWiki familiarity would be great!
• Developing Free and Open Source software, or being part of an open-source community

The Wikimedia Foundation is... 

...the nonprofit organization that hosts and operates Wikipedia and the other Wikimedia free knowledge projects. Our vision is a world in which every single human can freely share in the sum of all knowledge. We believe that everyone has the potential to contribute something to our shared knowledge, and that everyone should be able to access that knowledge, free of interference. We host the Wikimedia projects, build software experiences for reading, contributing, and sharing Wikimedia content, support the volunteer communities and partners who make Wikimedia possible, and advocate for policies that enable Wikimedia and free knowledge to thrive. The Wikimedia Foundation is a charitable, not-for-profit organization that relies on donations. We receive financial support from millions of individuals around the world, with an average donation of about $15. We also receive donations through institutional grants and gifts. The Wikimedia Foundation is a United States 501(c)(3) tax-exempt organization with offices in San Francisco, California, USA.

As an equal opportunity employer, the Wikimedia Foundation values having a diverse workforce and continuously strives to maintain an inclusive and equitable workplace. We encourage people with a diverse range of backgrounds to apply. We do not discriminate against any person based upon their race, traits historically associated with race, religion, color, national origin, sex, pregnancy or related medical conditions, parental status, sexual orientation, gender identity, gender expression, age, status as a protected veteran, status as an individual with a disability, genetic information, or any other legally protected characteristics.

If you are a qualified applicant requiring assistance or an accommodation to complete any step of the application process due to a disability, you may contact us at recruiting@wikimedia.org or (415) 839-6885.

U.S. Benefits &Perks*

• Fully paid medical, dental and vision coverage for employees and their eligible families (yes, fully paid premiums!)
• The Wellness Program provides reimbursement for mind, body and soul activities such as fitness memberships, baby sitting, continuing education and much more
• The 401(k) retirement plan offers matched contributions at 4% of annual salary
• Flexible and generous time off - vacation, sick and volunteer days, plus 22 paid holidays - including the last week of the year.
• Family friendly! 100% paid new parent leave for seven weeks plus an additional five weeks for pregnancy, flexible options to phase back in after leave, fully equipped lactation room.
• For those emergency moments - long and short term disability, life insurance (2x salary) and an employee assistance program
• Pre-tax savings plans for health care, child care, elder care, public transportation and parking expenses
• Telecommuting and flexible work schedules available
• Appropriate fuel for thinking and coding (aka, a pantry full of treats) and monthly massages to help staff relax
• Great colleagues - diverse staff and contractors speaking dozens of languages from around the world, fantastic intellectual discourse, mission-driven and intensely passionate people

*Please note that for remote roles located outside of the U.S., we defer to our PEO to ensure alignment with local labor laws.