Helping Moms make purchase decisions

zulily’s unique needs

“Something special every day”  is our motto at zulily. Thousands of new products go live every morning. Most of these products are available for 72 hours and a good number of them often sell out before the sales event ends! Many of our engaged customers visit us every single day to discover what’s new. We want to make sure our customers don’t miss out on any items or events that may be special to them, while also giving them more confidence in their purchase decisions. A traditional eCommerce ratings and reviews model could help, but is not the best fit for zulily’s unique business model which offers customers a daily assortment of new products, for a limited amount of time. We needed a different approach.

Our solution

Our specific business requirements demanded a more real time and community-oriented approach. We came up with a set of signals that highlighted the social proof and scarcity. Signals like “Only X left” and “Almost Gone” were designed to encourage users to act on a product that they are interested in before it is gone. Signals like “Popular”, “X people viewing” and “Y just sold” were intended to give users more confidence in their purchase decision. We were quickly able to bring these signals to life, thanks to our real time big data platform. These signals were shown on our product pages and the shopping cart.

Product page

Shopping cart

Results

We tested the feature out on our web and m-web experiences. The results turned out to be better than our most optimistic expectations! It was interesting to note that the feature was almost twice as effective on the mobile device compared to desktop. In hindsight it made a lot of sense as our customers have a lot of small sessions on the mobile devices during the day and this information helped them make timely decisions. The social and scarcity signals turned out to be a perfect complement to zulily’s unique business model.

zulily’s Kubernetes launch presentation at OSCON

In July Steve Reed from zulily presented at O’Reilly’s Open Source Convention (OSCON). He spoke to zulily’s pre-launch experience with Kubernetes. It was an honor for zulily to be asked to speak as part of the Kubernetes customer showcase, given the success we have had with Kubernetes.

Kubernetes launch announcement: http://googlecloudplatform.blogspot.com/2015/07/Kubernetes-V1-Released.html

Partnering with zulily – A Facebook Perspective

During F8, Facebook’s global developer conference (March 25-26, 2015), Facebook announced a new program with zulily as one of the launch partners: Messenger for Business. Just a month later, the service was live and in production! We on the zulily engineering team truly enjoyed the partnership with Facebook on this exciting product launch. It’s been a few months since the launch and we reached out to Rob Daniel, senior Product Manager at Messenger, to share his team’s experience on collaborating with zulily.

Here’s what Rob had to say:

How Messenger for Business Started

Messenger serves over 700 million active people with the mission of reinventing and improving everyday communication. We invest in performance, reliability, and the core aspects of a trusted and fast communication tool. Furthermore, we build features that allow people to better express themselves to their friends and contacts, including through stickers, free video and voice calling, GIFs, and other rich media content. We build the product to be as inclusive as possible – even supporting people that prefer to register for Messenger without a Facebook account – because we know that the most important feature of a messaging app is that all your friends and anyone you’d want to speak with is reachable, instantly.

In that vein, we found that some of people’s most important interactions are not just with friends and family, but with businesses they care about and interact with almost every day. However, until recently Messenger didn’t fully support a robust channel for communicating with these businesses. And when we looked at how people interact with businesses today – email, phone (typically…IVR phone trees), text – there were obvious drawbacks. Either the channels were nosiy (email), or lacked richeness (text), or were just really heavyweight and inefficient (voice).

Reinventing How People Communicate with Businesses

When we started talking with zulily about a partnership to reinvent how people communicate with businesses, under the mission of creating a delightful and high utility experience, we found we met an incredible match.

Why? Simply put, zulily thinks about Mom first. That’s the same people-centric approach to building products that’s at the heart of Messenger. And along the way, as we scoped out the interactions for launch, it was evident that we had the same mindset: build a product with incredible utility and value to people, establish a relationship and build trust, and over time build interactions that pay dividends via higher customer loyalty and lifetime value. And despite time pressures and the many opportunities to put business values over people values, we collectively held the line that people were first, business was second.

We also found that “zulily Speed” was no joke, and matched the Facebook mantra of “Move Fast.” Up and through our announcement at F8 (Facebook’s two-day developer conference) and launch later the next month, our two teams moved in sync at incredible speed. As an example, late one evening an issue was spotted by a Messenger engineer and by 10:00 am the next morning the problem had been identified, fixed and pushed to production. That kind of turn around time and coordination is just unheard of between companies at zulily and Facebook’s scale.

Speaking to the strength of the individual engineers on both sides, team sizes were small. This led to quick decision making and efficient, frequent communications between teams, forming a unique bond of trust and transparency. Despite the distance between Seattle and Menlo Park, we rarely had timing miscues and latency was incredibly low.

This Journey is 1% Finished

At Facebook we say “this journey is 1% finished,” highlighting that regardless of our past accomplishments we have a long journey ahead, and that we owe people that use our services so much more. And in that same spirit, we respect that Messenger is only beginning to provide zulily and other businesses the communication capabilities they need to form lasting, trusted, and valuable relationships with their customers. But we’re thrilled to be teamed up with a company, like zulily, that has the commitment and vision alignment to help shape the experience along the way and see the opportunity along the path ahead.

zulily hosted Women Who Code Event

WomenWhoCode_003

A sample of zulily’s “women in tech”

zulily knows that women in tech rock! On January 28, 2015 zulily hosted the Women Who Code Seattle chapter’s kick-off meeting, bringing women in technology together to discuss their passions. zulily has built its business on providing great value to moms. Our female engineers are a big driving force in delivering this expectation, so we were very excited to partner with  Women Who Code.  

Women Who Code is a global nonprofit organization that supports women in technology, helping them advance their careers. They provide supportive environments where women can learn from each other while offering mentorship and networking. 

The night started with mingling in a variety of technology discussions ranging from software, robotics, data analysis and bio engineering! College students and seasoned professionals gathered to share experiences, answer questions and consume food and wine. The atmosphere was inspiring and empowering. 

Kristin Smith, CEO of Code Fellows, shared with us her story and career path from Amazon to zulily to CEO of Code Fellows. Kristin talked about how women should be persistent in constantly learning new things and trying new technologies to empower their skills and overcome their self-doubt. “Embrace the ambiguity. Embrace the fear. That’s the time when things are going to explode!” 

WomenWhoCode2015_092

Kristin Smith, CEO of Code Fellows

We continued the night with interesting tech talks from women who work in zulily tech. Jaime Dughi is Sr. Product Manager of zulily’s Mobile Development team. She walked us through zulily’s unique business model and core development principles: fearless innovation and moving fast. She described the future of the mobile app platform in zulily’s business and how her team is advancing mobile development to tackle the demands of zulily customers while embracing the ever-changing mobile app landscape. (slides)

WomenWhoCode2015_122

Jaime Dughi – Sr. Product Manager, zulily Mobile Development Team

Our next speaker was Sara Adineh, Software Engineer on the Personalization Team. Sara remarked on her passion in bringing more women in tech, and shared her past volunteer activities: helping run a club for female students at her school to support women in the tech and science fields. She dove deep into what the Personalization Team does at zulily. “Personalization is the heart of zulily’s business model, presenting something special every day for each member” said Sara. She described the mathematics and Machine Learning techniques that zulily uses to understand what zulily’s customers need, so we can provide them something fresh every day. Sara talked about one of the engineering principles of zulily’s tech culture: zulily had open-sourced some of its projects as well as contributed to open-source technologies (such as Go, Kubernetes, and NSQ) used by the company. (Slides)

WomenWhoCode2015_189

Sara Adineh – Software Engineer, zulily Personalization Team

Our last speaker was Echo Li, Data Engineer on the Business Intelligence and Data Services team at zulily. She started with “I’m a woman, I’m a mom, I’m an engineer and I’m proud!” and the crowd went wild! She continued explaining her work as a data engineer at zulily, and spoke on how to understand data and get meaningful information out of it. She talked about different stacks of the data-processing pipeline and analytics here at zulily. She shared with us the growth of zulily’s big data platform from SQL server to Hadoop, Google Cloud and Big Query. (Slides)

WomenWhoCode2015_209

Echo Li – Sr. Data Warehouse Engineer, BI and Data Services

It was a night of thoughtful questions, engaged attendees and new friendships made. You can find it on the comments. We are looking forward to hosting the next Women in Tech event! Follow us if you want to hear about our future events!

WomenWhoCode2015_109

WomenWhoCode2015_167

WomenWhoCode2015_013

Facebook–zulily Joint Hack Day

zulily launches over 9,000 unique styles – more than the size of an average Costco – every day. This causes unique time and scale challenges for the Marketing team as they rapidly create, place and manage ads. Facebook and zulily’s Business and Technology teams have been working together to build a platform – the Acquisition Marketing Platform (AMP) – to automate the ad management process on Facebook.

During this partnership, we realized that both companies share a passion for enabling the engineering teams to move fast and put new, exciting features in the hands of customers. In that spirit, we hosted a joint hack day to see how we can further advance the AMP toolset.

FullSizeRender[1]
Joint FB-ZU Hack day participants: David Whitney (FB), Leo Hu (ZU) and Tim Moon (ZU). Not in picture: Omar Zayat (FB), Justin Sadoski (FB), Mason Bryant (ZU) and Rahul Srivastava (ZU)

The key wins for the hack day:

  • End-to-end automation: Currently, Marketing uses AMP to create the ad assets and Facebook’s PowerEditor to target and place the ads. The joint team built a prototype, utilizing the FB Graph and Ads APIs, to automate the ad process entirely within AMP. This is a huge win for the team! Marketing no longer has to switch between tools and now has a central place to manage all FB ads. This feature will further enable Marketing to scale up their ad creation process. The AMP Engineering team is now working on building a production version of the prototype.
  • Ad objects hierarchy management: FB ads have a hierarchy of campaign, ad-set and ad. The joint team spent time diving deep into the API to manage this hierarchy. Building this understanding is critical to automating management of campaigns, ad sets and reporting on key metrics.
  • API-driven ad Targeting: There are many ways to define the target of an ad on FB: interest, website customer audience, custom audience, etc. The team used the Targeting API to dynamically change the target range of an ad. This enables zulily to build some really interesting scenarios that can leverage real-time data in its systems; as we track the cost and conversion of every ad that is launched, we can now automate the increase or decrease the targeting reach via the API calls.
  • Account management: In October, FB released a new model to manage business ad accounts. The team spent time revamping the existing account model. This was a huge help and removed numerous access- and account-related roadblocks to further development. It would have taken us days of back-and-forth over email and conference calls to adopt the new model. During the hack day, this was completed in hours!

We had a fun and highly productive day-and-a-half of hacking. It was amazing to see the cool ideas and products that materialized from this event. The AMP team is super excited (and the Marketing team even moreso!) to build all the features that are now on their backlog. We are looking forward to future FB-ZU hack days!

Expect: How a 20-Year-Old Tool Saved My Project

I joined zulily in August of 2010, and at the time the company as a whole consisted of only 35 people. One of my first projects involved integrating one of our systems with a system owned by a much larger, more established partner company. The details of what these systems did aren’t relevant, except that the integration mechanism was for our system to drop XML files on an SFTP server that the partner company owned and operated.

At the time we were a 100% PHP shop (this has since changed), so I implemented our side of the integration in PHP, and used an open source PHP library called phpseclib to handle the actual SFTP data transfer. The partner company didn’t have any clients who used PHP, and didn’t officially support this library, but it worked great throughout development and integration testing. The integration test phase of the project took approximately 3 months, and not once during that time did we ever have even the slightest hiccup when transferring data between systems.

However, once we went to full production, we started seeing file corruption — specifically, sometimes files we transferred to the partner’s SFTP server would be truncated. There was no discernible pattern to the truncation; it happened at different points every time, and often a file that failed once would work fine when it was retried.

Naturally, this caused some consternation, as our code hadn’t changed, and it had been working for months without fail. When I pointed the exact same code at their test server, and sent the exact same file content, everything worked flawlessly. When I pointed it back to their production server, the files were truncated.

Clearly, to my mind at least, the problem was on their end. After a couple of days of frenzied troubleshooting, we discovered that the version of the SFTP server software running on their test machines was newer than what they ran in production. Presumably, we were hitting some bug in that software that was fixed between the two versions.

Since they were a very large company with many other clients using these servers, they were unwilling to upgrade their SFTP software on our behalf. Also, given that we were already well into the go-live phase of the integration, rewriting our system in another language wasn’t an appealing option, especially since there was no guarantee that the new implementation would work any better.

One option that the parent company did officially support, though, was the sftp client included with OpenSSL. I tried manually transferring a few files this way…and the file truncation issue disappeared.

There were problems with this approach, though: for one, the SFTP server required authentication, and the partner company was unwilling to set up SSH keys for us to do non-interactive authentication. OpenSSL’s sftp client doesn’t support setting the authentication credentials via command line parameters, leaving us stuck with authenticating interactively. This obviously wasn’t an acceptable long-term solution, since these systems needed to communicate with each other without human intervention.

I don’t recall exactly when I stumbled across it, but somewhere in the midst of searching for a solution I came across Expect.

Quoting the introduction on the Expect homepage:

Expect is a tool for automating interactive applications such as telnet, ftp, passwd, fsck, rlogin, tip, etc. Expect really makes this stuff trivial.

This sounded exactly like what I needed! I bought a copy of “Exploring Expect“, read enough of it on the train ride home to get started, and after a couple of hours I had whipped up a PHP script that did the following:

  1. Dump the XML data we wanted to transfer into a local temp file.
  2. Build an Expect script in memory by doing some variable interpolation in a PHP string.
  3. Write the resulting Expect script to another local temp file.
  4. Exec() the Expect script, capturing the process return code and anything the script wrote to stdout or stderr.
  5. Write the process output to our application log for later troubleshooting if anything went wrong.
  6. Finally, if the process return code was zero (indicating success), delete the two temp files. Otherwise, send an email to our notification alias so I could investigate.

Here’s the meat of the Expect script generation in all its glory:

$expectContents =
 '#!/usr/bin/expect' . "\n"
 . 'set timeout ' . $this->m_timeout . "\n"
 . 'spawn sftp -o Port=' . $this->m_port . ' ' . $this->m_user . '@' . $this->m_host . "\n"
 . 'expect {' . "\n"
 . ' default {exit 1}' . "\n"
 . ' "Connecting to ' . $this->m_host . '..."' . "\n"
 . '}' . "\n"
 . 'expect {' . "\n"
 . ' default {exit 2}' . "\n"
 . ' "continue connecting (yes/no)?" {send "yes\n"; exp_continue}' . "\n"
 . ' "ssword"' . "\n"
 . '}' . "\n"
 . 'send "' . $this->m_pass . '\n"' . "\n"
 . 'expect {' . "\n"
 . ' default {exit 3}' . "\n"
 . ' "ermission denied" {exit 4}' . "\n"
 . ' "sftp>"' . "\n"
 . '}' . "\n"
 . 'send "cd /inbound\n"' . "\n"
 . 'expect {' . "\n"
 . ' default {exit 5}' . "\n"
 . ' "not found" {exit 6}' . "\n"
 . ' "No such file" {exit 7}' . "\n"
 . ' "sftp>"' . "\n"
 . '}' . "\n"
 . 'send "put ' . $filename . '\n"' . "\n"
 . 'expect {' . "\n"
 . ' default {exit 8}' . "\n"
 . ' "not found" {exit 9}' . "\n"
 . ' "sftp>"' . "\n"
 . '}' . "\n"
 . 'send "quit\n"' . "\n"
 . 'exit 0' . "\n";

Yes, I’m aware that there are more elegant ways to interpolate strings in PHP than this, but at the time I was still fairly new to PHP and under a ton of pressure to get something — anything — working.

Coming from a background in statically typed compiled languages, this just felt wrong somehow. Even my stints in the land of Perl didn’t feel this hacky. It was an ugly, nasty, weird abomination…but it worked.

And it kept working, without fail, for the entire lifespan of this system. Over time, we encountered bugs in many other areas, as is inevitable with any complex system, but this little corner of the codebase never had a single issue.

What this taught me is that sometimes you should go slowly, be methodical, and take the time necessary to create a simple, elegant solution to your problem.

And sometimes you just need to break out the duct tape.

welcome to the zulily engineering blog!

It has been just over four and a half years now since Darrell and Mark (our two co-founders) came up with the original idea for zulily.  And from the beginning we’ve focused on building software to power a new way of shopping online.  We call it discovery-based shopping.

Here at zulily our tech team is at the core of the business and involved in the entire life-cycle of both our vendors and our customers.  Whether it’s building internal tools for our merchandizing and studio teams, launching new features on our vendor portal or vendor data exchange or delivering a new personalized experience on our mobile or site experience, we are always focused on challenging ourselves to build world-class solutions which exceed expectations.

We are a build shop and big supporters of the open source community.  We believe in the power of the community and feel we have an obligation to give back to the projects that have helped us get to where we are today.  As we continue our transition from small, frenetic start-up, expect to see us continue to be more active in the community.

At our core we have 10 values we try to live by on a daily basis.  These have served us well over the past 4+ years as we’ve tried new things and experienced major wins… and a number of “well, that was a bad idea” moments.

  1. “No” is not in our vocabulary — we strive to find creative solutions and the path to “yeah, we’ll give it a go”.
  2. We believe in speed of innovation and taking agile development to the extreme.
  3. We embrace a customer-centric view to delivering technology solutions — always start with the customer.
  4. Mistakes are expected and encouraged — we learn from them and move on.
  5. We empower our engineers to solve business problems and tailor our process accordingly.
  6. Engineers write production code and own it from start to finish.
  7. We are defensive in nature: we assume things will break and plan for it.
  8. We believe in “just-in-time” software with an eye towards capacity and scalability.
  9. We value full transparency and continuous communication.
  10. We strive to find the simple solution in anything we do.

In the end we’re all about building an amazing team, passionate about building awesome software and technology solutions.  We love to move fast and take risks.  And we’re big believers in the idea of continuous improvement.

Thanks for taking a few minutes out of your busy day to read our tech blog.   We hope you enjoy it!

Luke