AUSTIN, TX – With more than 200 digital commerce searches each year, EcommerceRecruiter.com is the leading contingency-based executive search firm serving the Shop.org, Ad:Tech, and IR-500 communities. To opt-in to our popular “Ecommerce Job of the Day” email, click here. To invite Harry Joiner to connect on LinkedIn, use Harry.Joiner@EcommerceRecruiter.com
One of the coolest Big Data jobs around …
We are working with RetailMeNot in their search for a Senior Analyst, A/B Testing – Statistical Analysis based in Austin, TX. RetailMeNot, Inc operates the world’s largest digital coupon marketplace. The company enables consumers across the globe seeking to save money to find hundreds of thousands of digital coupons from retailers.
RetailMeNot experienced more than 500 million visits to its websites in the last 12 months. The RetailMeNot portfolio includes RetailMeNot.com, the largest digital coupon marketplace in the United States; RetailMeNot.ca in Canada; VoucherCodes.co.uk, the largest digital coupon marketplace in the United Kingdom; Deals.com in Germany; Actiepagina.nl, a leading digital coupon site in the Netherlands; Bons-de-Reduction.com and Ma-Reduc.com, leading digital coupon sites in France; Poulpeo.com, a leading digital coupon site with cash back in France; and Deals2Buy.com, a leading discount offer site in North America. RetailMeNot went public in July 2013 and is listed on the NASDAQ stock exchange under the ticker symbol “SALE.”
RetailMeNot is looking for a talented and experienced Senior Analyst to support its A/B testing statistical analysis methods. The ideal candidate will be responsible for three areas …
- Assessing test hypotheses and evaluation potential opportunity and validity;
- Effectively utilizing site traffic that meets AB test demand and build test monitoring models/alerts to reduce financial impacts during AB test period;
- Standardizing statistical methods in analyzing AB test results and provide actionable findings.
Through leveraging statistical frameworks; technical expertise in data models and analytical skills this Senior Analyst will ensure that the AB test analysis results produces accurate findings on changes to product functionality, merchandising content, personalization algorithm and SEO optimization focusing on desktop, mobile web and mobile native app platforms.
This Senior Analyst must develop a detailed understanding of the Company’s A/B testing operations as well as a thorough knowledge of available analytics & data warehousing solutions. The Senior Analyst will be integral part of the Global A/B Testing function which is part of Global Analytics team. This Global Analytics group’s efforts are highly visible and engaged across various groups within the organization and rolls up to CFO of RetailMeNot.
- You’re curious.
- You’re naturally curious and have an inquisitive nature.
- You’re not intimidated by vast amounts of data.
The firm’s business continues to grow, consider these numbers from 2012:
- 450 million site visits
- 160 billion rows of data
- 9 million newsletter/alert subscribers
- 5 million mobile application downloads
- 2.5 million Facebook likes
- 160K Twitter followers
- 89 of Internet Retailer 100 are business partners
- 20% of total site traffic is from mobile
- 7 global sites
- 1 central Global Analytics team
HARRY’S COMMENTS: RetailMeNot is the digital coupon marketplace into which Austin Ventures, Google Ventures, Adams Street Partners, Norwest Venture Partners, IVP and JPMorgan poured more than $300 million. The firm’s portfolio of coupon aggregation websites were visited 500+ million times in 2013, and its iPhone app reached #2 in the App Store and remained in the top 20 overall in December. And get this: RetailMeNot is the only shopping app in the top 50, ahead of Amazon and Walgreens.
With so much traffic, it’s easy to understand why RetailMeNot is looking to add top-flight statistical analysis talent to its web analytics team. The firm seeks to build out its A/B testing capability, moving away from a situation in which multiple people shared that responsibility. With the role having been split among multiple team members, A/B test results were simply not being analyzed to an optimal degree. This position will be responsible for managing those tests more rigorously and accurately, and adding resources where appropriate. Basically, RetailMeNot is looking to ramp up the volume of testing across all its websites.
The basic expectation for this new Senior Analyst is that s/he will establish an analytical framework for the team to use, making sure that all appropriate stakeholders are getting what they need out of RetailMeNot’s testing and analysis process and can take the analysis and insights into their own decision-making process. It’s all about setting a standard of analysis for the company.
It all starts with asking the right questions.
This Senior Analyst will help RetailMeNot structure those inquiries with a standard set of tools and dashboards to help solve and monitor testing capabilities. In three months, RetailMeNot wants this Senior Analyst to show value in the program, partially by building trust in the data. S/he will be the team’s statistician, and it will be up to them to track all the right data, and understand what tags are applied to what, when, where, how, and why.
Once the data is gathered, it must be structured in a way that gives all key data meaning. (Basically, if the data is structured correctly, then it cannot be pulled incorrectly.) Additionally, a process for monitoring changes must be developed and implemented so that if any problems or issues occur later, the right people are notified immediately and the proper solution is recommended. Once the test data is gathered, the Senior Analyst must have the right approach to analyzing it. Clearly, this is a very technical position, and it’s essential that the new hire understand the nitty-gritty nuances of data and what it represents.
RetailMeNot needs this role’s help in assessing how long tests will take; how big the sample sizes must be; as well as what are the approaches the firm should take to set up tests. For example, should a test involve two variables? Or should it be multivariate? The Senior Analyst will be steeped in quantitative issues, looking at the data, making sure that the data is correct, that the team is using the data correctly, and pulling the correct data correctly.
Data!! Data!! Data!!
The Senior Analyst must be very hands-on and extremely knowledgeable on all aspects of data collection and management. S/he Must know how to write SQL statements and understand where all the data resides in RetailMeNot’s business systems. Sometimes the data will live in 2 different systems. Sometimes it’s more involved. For example, RetailMeNot uses Google Analytics as well as SiteCatalyst in 32 different places. Different software calculates the same web visits differently, and the Senior Analyst will need to tease out how these differences will impact RetailMeNot’s approach to managing its business.
Is this job right for you?
This position is perfect for somebody who loves (what else?) DATA him and can’t get enough of it. The ideal Senior Analyst will love to structure data; clean data; and organize massive amounts of data to develop ways to drive interesting insights from their rigorous analysis of it. Naturally, s/he will be extremely technical, and won’t be intimidated by how data is gathered. Obviously, it’s essential to have some programming skill to automate some of the tests that would be tedious for a non-technical person (like me) to perform.
Beyond loving to work with data, it’s essential that the Senior Analyst have the ability to communicate with the business intelligence team. S/he must be able to take complex statistical concepts and terms and dumb them down into plain English. It’s all about making these insights more accessible to everyone on the team. If you think you can do that in a way that brings fun to your job, that we would certainly love to hear from you.
What are you going to do in this role?
Hypothesis Validation (25%)
- Hypothesis validations – Participate in hypothesis generation process and clearly validate opportunities/recommend actions that can have positive improvements in our site experience and financials; help stakeholder teams to think about site opportunities in a different way by asking questions; provide feedback to stakeholders to influence test ideas that will yield the most impactful customer experience improvements on the site
- Calculate traffic allocation – Understand traffic baselines for pages being tested, identify traffic required to achieve significance; effectively use page traffic for various concurrent tests
Data Models (25%)
- Data Models – Develop data models to improve financial efficiency of testing program. These models could be used to identify customer cohorts to target, site wide financial impact on pre/post tested feature roll-outs, stop-loss model to assess financial impacts on when to stop tests, predict long term trends on short term metrics from AB tests, etc.,
- Test validation – Understand test setup and data collection standards within test environment and provide feedback to product/engineers on preliminary test set-up findings before test starts
- Set-up alerts – Ability to query databases to understand current baseline and set-up test monitoring alerts to detect trends if it exceeds upper/lower limit thresholds; provide initial trend alerts to stakeholders confirming test setup is accurate in production
Analysis & Insights (50%)
- Establish Test KPI baselines – Identify conversion and engagement KPI baselines before the test starts and analyze results during and post-test timeframes
- Test cohort analysis – Drive proper AB or multi-variant testing analysis with statistical rigor and perform cohort studies to answer “why?” on test result drivers. Support more complex tests from an analytical and reporting perspective
- Qualitative side of test results – Incorporate qualitative survey results in test results analysis to understand customer behavior and satisfaction scores better
- Leverage supporting tools – Home-grown tools (such as Beacon data collection; 3rd party tools such as Google Analytics, Adobe SiteCatalyst, Adobe Target); statistical analysis package (such as R or SAS); data query methods (using SQL or Hive; Excel Macros) and visualization tools (such as Tableau) to inform analytically-driven recommendations
- Deliver insights – Objectively deliver testing insights and help create a culture of data-driven decision making
Experience / Requirements:
- Minimum 3 years of eCommerce AB testing focused statistical analysis , site optimization and/or web analytics experience or in a related data analytics field (financial analysis, digital analysis, statistical modeling)
- Bachelor’s degree in statistics, finance, mathematics, economics, or in a quantitative discipline; Master’s degree preferred
- Solid understanding of online analytic metrics and related statistical concepts. Experience with statistical analysis package (such as R, SAS or SPSS); web analytics tools (such as Adobe SiteCatalyst / Omniture, Google Analytics); A/B or Multi-variant testing tools (such as Adobe Target, SiteSpect or home grown testing tools) is a must
- High-level proficiency with statistical methods; SQL based query languages; relational database concepts and Microsoft Excel Macros
- Excellent analytical skills – ability to understand and interpret data within the context of the AB testing – solve problems and distill data into actionable recommendations
- Excellent communication skills – this position requires confidence in directly working with senior leaders in the organization and will be tasked to communicate complex test results findings and recommendations persuasively, in a compelling and easy to understand manner
- Demonstrated ability to work with cross-functional teams; independently and proactively
- Intellectual and analytical curiosity – initiative to dig into the “why” of various results
- Excellent project management skills including the ability to formulate and identify key performance indicators for tests and drive detailed test results
- Flexible and responsive, ability to prioritize multiple tasks, reprioritize as needed and work with a sense of urgency to meet deadlines in a fast-paced, dynamic work environment
- Desire to learn new technologies and grow analytical skillset
- Intellectual curiosity, passion for problem-solving, and comfort with ambiguity
- Rigorously detail-oriented with commitment to accuracy
Ideal Candidate has experience with:
- Master’s degree in statistics, finance, mathematics, economics, or in a quantitative discipline preferred
- Minimum 3 years of eCommerce AB testing focused statistical analysis , site optimization and/or web analytics experience or in a related data analytics field (financial analysis, digital analysis, statistical modeling)
- Hands-on experience in statistical analysis package such as R, SAS or SPSS; high-level proficiency with SQL based query languages; relational data concepts and Microsoft Excel is a must
- Excellent analytical & communication skills – ability to understand and interpret data within the context of the AB testing – initiative to dig into the “why” of various results; communicate complex test results findings into actionable recommendations
UPDATE: THIS SEARCH IS CLOSED.
PS – If you’re thinking about changing jobs this year (though us or anyone else), you might want to check out this post on why even TOP candidates get rejected.
