Experimentation and Analysis Manager - Paid Media Experimentation - Remote

As an Experimentation & Analysis Manager, you'll use your combination of marketing expertise and analytical skills to run growth experiments, analyze the results, identify impactful growth opportunities, and educate growth teams on rigorous experimentation. You'll work with the Growth Strategy & Operations and Growth Data Science and Engineering teams to conceptualize and build the foundation for experimentation at Shopify.

We care about growing creative, collaborative people, and strong, capable teams. Your peers in Shopify's Growth Org have diverse backgrounds, skill sets, and experiences but share an established track record of driving significant business growth.

The Experimentation & Analysis Manager will be responsible for embedding within paid channel Growth teams to lead all parts of the experimentation process, from hypothesis generation and experimental design to statistical analysis and internal education on rigorous experimentation practices. Your success will hinge on your ability to strategically design paid ad experiments across many platforms, rigorously analyze and present the results, and enable your colleagues through best practices and streamlined workflows.

You'll be doing things like:

• Strategize and lead the experimentation roadmaps for high-growth paid channel teams. Conceptualizing, running, and analyzing experiments both on-site and in-ads.
• Collaborate with conversion rate optimizers and paid marketers in Growth to help them run smarter experiments, faster.
• Deep dive on experimentation workflows, educating and enabling your colleagues to run more rigorous experiments.
• Work, as part of a multidisciplinary team, on answering questions and unlocking growth through experimentation.
• Use your analytical skills to find new opportunities the company should be exploring and experimenting with.
• Influence the roadmaps of other teams and help craft the future direction of the company.
  

Qualifications

• Expertise in experimental design and analysis, including A/B, and full and partial factorial designs that control for threats to validity. Experience connecting ad-level metrics and performance to long-term effects on leads and customers.
• Experience working in performance marketing channels, such as AdWords, Facebook, YouTube, etc. You need to have domain expertise and understand the nuances of how each channel operates and how each channel's experimentation platform operates.
• Experience with commonpaid experimentation methodologies, such as geo experiments, PSA/placebo tests, ghost ads, etc.
• Strong experience working with high-velocity experimentation programs. You're familiar with the entire experimentation process, from ideation through to iteration.
• Experience with analytics tools, such as Excel, Mode, Tableau, Amplitude, and/or Google Analytics.
• Strong communication and leadership skills, including setting an experimentation strategy and designing experimentation workflows, managing stakeholders' requests, etc.
• Experience with Bayesian statistics and mathematical concepts, such as sample size, credible intervals, metric grain, etc.
• Experience working with different stakeholders across multiple disciplines, including data scientists, engineers, paid channel representatives, and marketers.

It would be wonderful if you also had:

• Experience working in SaaS or other technology companies. It's useful to understand our industry and business model.
• Experience with SQL and/or Python. You may not be able to write complex queries from scratch, but you have a high-level understanding of databases and how to query them.
• Strong project management and prioritization skills with an approach that defaults to action. You can drive good decisions quickly and help other teams do the same.
  

Shopify is now permanently remote, and working towards a future that is digital by default. Learn more about what this can mean for you.

Our belief is that a strong commitment to diversity & inclusion enables us to truly make commerce better for everyone. We encourage applications from Indigenous peoples, racialized people, people with disabilities, people from gender and sexually diverse communities, and/or people with intersectional identities. Please take a look at our Sustainability Reports to learn more about Shopify's commitments to our communities, and our planet.

At Shopify, we understand that experience comes in many forms. We're dedicated to adding new perspectives to the team - so if your experience is this close to what we're looking for, please consider applying.

How we hire

At Shopify, we put a lot of care and time into who we hire. We believe that in order to build the best products, we need to build high impact teams. Our recruitment process centres around what we call the Life Story interview, a conversational-style interview where we get to learn more about you.
Learn more about our hiring process