Why Don't They Add Up? Understanding How Web Metrics Work Across Time Periods

It’s a common question, really.  Inevitably someone will walk up to you and say, “There is something wrong with your reporting.  When I add up your daily visitors they do not equal the weekly visitors you are reporting.  Then, I add up the monthly visitors and they do not equal the yearly visitors.  Why are you underreporting?”

Of course, you are not underreporting at all, and Visitors is not the only metric impacted by this phenomenon.  This is a great time to grab a white board and explain.  

Web analytics tools gather data on web activity 24/7 in a constant stream.  This stream is then portioned into time buckets, for example: daily, weekly, monthly, quarterly and yearly.

To do this, the data is cut at the time cutoffs for each bucket and then totaled within the bucket to calculate metrics like: hourly visitors, daily visitors, weekly visitors, monthly visitors, quarterly visitors and yearly visitors. It is the cutting that could place a single visitor into two time buckets at the same time, which would double-count if the two time periods were added together, and when the data is uncut, the single visitor is counted only once.

The best way to explain is to illustrate with an example.  It is Saturday, June 30th at 11:50PM and I visit karenlynnvincent.com.  While I am there, I read the About Karen Vincent page and 3 of her blog entries, for a total of 4 pages.  In the middle of the last entry I am reading, I notice that it is now Sunday, July 1 at 12:03AM.  I shut down my computer and go to sleep.

Yearly Reports:  I am 1 Visitor in 1 Visit with 4 Page Views.  Yearly reports would reflect these metrics accurately.

Quarterly Reports:  The quarterly report would split my session at 12:00AM, July 1. This means that the Q2 Quarterly Report would show 1 Visitor, 1 Visit and 4 Page Views.  The Q3 Report would show 1 Visitor, 1 Visit, 1 Page View.  These reports are accurate for activity within each quarter, but would over-inflate what happened if the reports were added together.

Monthly Reports:  The monthly report would also split my session at 12:00AM, July 1.  The June report would show 1 Visitor, 1 Visit and 4 Page Views.  The July report would show 1 Visitor, 1 Visit, 1 Page View.  While the monthly reports accurately show web activity for each month, but when added together they would over-count activity.

Weekly Reports:  The weekly reports can be set to bucket weeks Sunday through Saturday or Monday through Sunday.  If bucketed Sunday through Saturday, one week would carry 1 Visitor, 1 Visit and 4 Page Views and the next week would have 1 Visitor, 1 Visit, and 1 Page View.  If bucketed Monday through Sunday, the report would keep the session together and would accurately reflect 1 Visitor in 1 Visit with 4 Page Views.

Daily Reports: Daily reports split at 12:00AM.  This means that Saturday would hold 1 Visitor, 1 Visit and 4 Page Views and Sunday would hold 1 Visitor, 1 Visit, and 1 Page View.  

Each report is accurate for its time period, but would provide inaccurate data if data for those time periods were to be added together manually in a spreadsheet.  To safeguard the accuracy of the reports, be sure that reporting you do for any time period is based on reports from your Web Analytics software for that time period, and that no manual adding is done.  Weekly reports should be based on Weekly blocks, Monthly on Monthly Blocks, etc to accurately reflect activity

Tell those who point out the difference to you that they are very clever to have found one of the lost secrets of data aggregation: that when it comes to aggregating, the whole is usually NOT the sum of its parts.  Encourage them to stop adding and instead pull fresh and accurate data for each time period they need.  This will provide a an accurate picture of activity on the site.

Single Page Visits: Is “Bounce Rate” Good or Bad?

A person comes to your website looks at one page and exits.  Whether this is good or bad depends entirely on your goals, and your goals will vary from page to page.  For this reason, using bounce-rate as a page-level metric can be helpful and as a site-wide measure can be misleading.  

Before we explore further, what do I mean by bounce?  A bounce occurs when a session ends after viewing only one page. It can be calculated in summary for the entire site by dividing single page visits by total visits or on the page level by dividing single page visits for that page by the total entry visits for that page.  Some analytics tools calculate bounce rate automatically, saving the time and trouble.

When is a High Bounce Rate is Good?

A high bounce rate is desirable when the goal of a page is to deliver information to the visitor, often to answer a specific question.  Once the answer is read, the visitor completes the task and does not need to view other pages.  A common page of this type is the Contact Us page where the visitor finds the phone number and accomplishes the task.  In this case, the faster you can help the visitor complete the task the better so seeing a bounce rate rise is representative of good performance on the page.  You may also see that the page is frequently bookmarked providing a strong repeat visitor base.

When is a Low Bounce Rate is Good?

On the other hand, a low bounce rate is desirable when the goal of a page is to generate leads, produce sales or drive engagement with content.   In this case, the goal of the page is to raise interest and drive interaction with the website so if the visitor moves on to another page on the site, then the page performed well.  For pages like these, driving toward a lower bounce rate is desirable.  

Recommendation

It is very helpful to put site-wide bounce rate to use as an index to pick up on any sharp movements with the introduction of new campaigns and new releases.  It can be a beacon when used internally and compared to its own past performance.

If you are able, define the goals on a page-by-page level and track the bounce rate for those pages.  This does not need to be done for the entire site, but it can be great for the handful of key pages.  If you find it is high when it should be low or low when it should be high, then you can begin to take steps with content and layout to help the page perform better.

Web Data Decisions for Auditing and Compliance: 4 Areas for Consideration

A good web application provides good experience on the front end for customers and the back end for web managers.  Project funding tends to focus on the front end experience primarily, but it’s really the back end that can break a business when something goes wrong.  

• A customer calls and claims they were given last year’s terms and conditions.  
• A product manager says IT uploaded the wrong PDF.  
• A customer says they never logged in to authorize that payment.  
• A customer says that the data they saw was not theirs.

Careful planning now will make auditing requests like these much less painful in the future.  

First, take steps to enhance security for your portal database.  Close all of the possible unauthorized ports of entry.  Ensure that firewalls are strong.  Check the latest security recommendations and do all that’s possible to meet each one.  Are there features to require additional authentication under circumstances where the user is out of his or her normal area or has failed some login attempts?

Now, check the design of the portal database to be sure that there are adequate auditing tables available.  How are transactions tracked? Is there a session detail record to view all activity during a session?  Is the user’s IP addresses from sessions recorded so that likelihood of identity can be considered?   A well-designed web portal database is an important part of auditing because it can include details of every transaction and activity by user, including text entered where necessary.  This is very helpful when a user calls into the call center with concerns about his or her account.

Next, prepare your Web Analytics software for auditing capabilities.  Work with your web analytics professional to pass an unintelligible user ID key to the web analytics software for logged in users and create a custom report to record pages and downloads by this user key.  This simple step is a godsend when something goes wrong, like a PDF or web page has an error in it and you need to quickly identify who has seen this content so that a correction can be made to that group of users.  This is extremely valuable from a risk management perspective.

Now, take an inventory of the tools you have.  Do you have any session recording capabilities?  It is great to have access to a tool that will allow you to replay any session in question to see exactly what pages were viewed and what was accomplished.  Most companies do not have access to these tools for their web properties, but tools like these can be very helpful for auditing, especially in industries that are strictly regulated.

After some preparation, answers to tough auditing questions become more routine.

• A customer calls and claims they were given last year’s terms and conditions.  Retrieve that customer’s encrypted ID, open the web analytics reporting and query which PDFs were viewed.  
• Then, a product manager says IT uploaded the wrong PDF.  In the web analytics package, query that PDF and view which customers, if any, saw the incorrect PDF so that the product manager can draft a letter to those customers.  
• Now, a customer says they never logged in to authorize that payment.  Use your web portal database to see details of that transaction, verify the session details to see if there was a chance the account was compromised.  
• If a customer says they saw data from another customer and you have a session recording tool, check the tool to see what happened.  If not, look into the web portal database session activity to see if any wires might have been crossed.

With careful planning and a few simple steps, you will be prepared for these and any other questions that come up in the future.  

Product Managers: Building Good Measurement into your Web Applications

With so much focus on appearance and functionality, web portal data decisions can be easily overlooked at a time when performance metrics are more crucial than ever.  As more functionality is added, good data collection is needed for product management.

Product managers need to understand how the site is being used to inform the product roadmaps and order website changes as needed.  There are two different building blocks for measuring the web performance and both are needed to achieve a complete understanding of the online customer: web analytics and web portal database.

Web Analytics

Web analytics tells you about the online behavior of the customer by painting a picture of the most popular content and how a customer moves through the website to accomplish tasks.  There is a wealth of information in the basic analytics reports found in web analytics packages like Google Analytics, WebTrends,  Site Catalyst, and CoreMetrics.  Even more information on fall out from web form fills, clicking to off-site links, and other detailed task and funnel behavior can be tracked with additional planning and page-specific javascript tags or URL parameters.

Web analytics is fundamental to all web properties, but is also commonly overlooked during the development process.  In order to determine the impact of the redesigned property, getting the proper code in place early in the project is essential.  The basic requirement is a snippet of code often referred to as a base tag.  Depending on the analytics package and the company policies, this is either a direct copy/paste or include to the page header or page footer.  This code is the engine that communicates page activity to the analytics package.  For advanced reporting and specialty metrics, a web analytics professional can provide code to be included within the content of the page to provide additional performance metrics.

Getting the basics is easy.  Often it only requires mentioning that you want the tracking installed, and in some cases, it is standard and automatically included with no effort at all.  For this reason, tracking is often taken for granted, and taking web analytics for granted is a huge mistake.  I have seen many cases at companies large and small where sites have gone live with no tracking because it wasn't mentioned in the business requirements. Without metrics, the product manager was not able to demonstrate the success of the product.

Add tracking requirements to the standard requirements template to be used in every web project and open discussions with web analytics professionals in your company at the start of the project.  This will ensure you can avoid KPI pitfalls when work is complete.

Portal Database

Make the most of the web portal database behind the application, used to deliver information to the page.  These databases run the application functionality of website, processing transactions, recording changes to user profiles, error messages delivered, logins, etc.  Because of this, they hold a wealth of information on how the site is being used.  While web analytics delivers web content and behavior details, the web portal database delivers information on adoption and usage at a transaction level.

The types of metrics you can expect to gain from the web portal database include: percent of users logged in in the last 90 days, login frequency, login duration, average transactions per person, total transactions, revenue, etc.   

The most common issue is overlooking the requirements for the performance indicators (KPIs) during the design and development of the database.  To be sure that the portal database will deliver the metrics you need, take an hour at the start of a project to brainstorm all possible performance indicators that my be needed in the future and discuss with the IT Architect and DBA on our team.  It is much easier to include these in the database design at the start than it is to make a change along the way.

Summary

The key to success is timing a three-step process: (1) consider the data you will need at the beginning of the redesign, (2) collect product management requirements early and (3) coordinate with DBA and Web Analytics professionals throughout the course of the project.  Early and continual involvement will ensure you are collecting good data from the start and can report your success measure KPIs along the way.