Holiday Data Spikes and Acquia CDP Platform Scalability

Learn how Acquia CDP handled data spikes and retail web traffic during Black Friday and the holiday shopping season.

One of the key differentiators of the Acquia customer data platform (CDP) is its scalability. Scalability is one of Acquia CDP's key architectural tenets, powered by our tech stack’s horizontal scalability and elasticity. This is evident from how large, high-demand enterprise brands deliver relevant and personalized customer experiences via the Acquia CDP, processing hundreds of millions of customer records, billions of transactions and billions of events daily. One illustration of Acquia CDP’s scalability is how easily we are able to add enormous volumes of historical data into the platform when onboarding each new enterprise client. 

Another key illustration of the Acquia CDP’s scalability is its ability to handle and ingest big spikes of data as we did during this season’s Black Friday and Cyber Monday for our retail clients. Let’s dive further into some interesting aspects of these data spikes. 

The CDP real-time pipeline ingests real-time events, including web events (browse as well as checkout/transactional events) and email events. We saw huge spikes across brands in these events and transactions, and our platform successfully ingested these data spikes. This success is not just about data ingestion but also about scalability in all the other modules like campaigns, analytics, and machine learning, which all need to operate on these increased volume spikes. Supporting spikes like this is mission critical to our clients’ bottom lines.   

Data Points in Web Traffic Volumes: 2019 Black Friday / Cyber Monday

  • Multiple brands had spikes of greater than 20 times their regular web traffic volume on Black Friday and again on Cyber Monday; these peaks correlate to the times when they sent large campaigns. Acquia CDP also ingested these spikes in email events, so the campaigns had double the impact on the data ingested into Acquia CDP.
  • One brand had spikes of greater than 30,000 events per minute (compared to their regular volume of less than 1,000). This was early morning on Black Friday.
  • Another brand sustained spikes over time resulting in greater than 1.3 million events per hour and greater than 25 million events per day on Black Friday.
  • Another company normally averages 20,000 calls per hour. But on the Sunday  ahead of Cyber Monday, it reached peaks of ~430K calls per hour, i.e., greater than 20 times their average.
  • Even some of our European clients saw about 20 times their normal volume consistently throughout the holiday weekend.
  • In that week, across brands, we processed more than 350 million web events — more calls in that week than an average month.

Data Points From Email Traffic Volumes

We saw spikes in email campaigns and the events generated by them, which are all ingested into Acquia CDP via the extensive ESP connectors we have. Some interesting data points are:

  • We ingested ~3.7 billion email events (which include email send, email open, email click) across all brands during the holiday month. That's 15% more than the email events in the previous month.
  • We've consistently seen higher email click rates, leading to better engagement during the holidays in 2018 and 2019. For example, in 2019, while 29% of the monthly email sends were in the holiday week, 32% of the monthly email clicks were during the holiday week thus showing greater engagement during holidays.
  • We supported multiple intra-day spikes for web and email events on the days of Black Friday and Cyber Monday. 

Data Points from Transactional Volumes

As expected with the increased web and email activity, this resulted in a spike in transactional data as well. We noticed some interesting data points in the transactional data ingestion

  • On Black Friday 2019, on average across brands, we ingested approximately six times our clients’ normal daily transactional volume across online and offline transactions. 
  • Some brands had greater than nine times their normal transactional volume on this day.
  • On average, our clients saw 13% growth in transactional volume when compared to the same period last year. A majority of that growth came from digital sales channels driven by convenient features such as "Buy Online; Pickup in Store."
  • The percentage of digital transactions in the total transaction volume during the week of Black Friday and Cyber Monday was 23% in 2018 then jumped up to 30% in 2019.

Powering Machine Learning, Campaigns, and Analytics

Once these data spikes are ingested through the Acquia CDP data pipeline, this data powers all the CDP modules, which illustrates the scalability in these modules as well. So, these data spikes directly translate to helping marketers achieve better results.

  • Acquia CDPs Machine Learning module generates greater than 1 billion predictions across brands
  • Acquia CDP's Metrics module powers dashboards and ad hoc reports to deliver insights based on this increased event and transaction volume.
  • Acquia CDP’s Actions module scales up to help marketers act on these newly ingested data spikes to deliver campaigns.

Featured Resources

View More Resources