Monitoring data comes in a variety of forms—some systems pour out data continuously and others only produce data when rare events occur. Some data is most useful for identifying problems; some is primarily valuable for investigating problems. This post covers which data to collect, and how to classify that data so that you can:

  1. Receive meaningful, automated alerts for potential problems
  2. Quickly investigate and get to the bottom of performance issues


Metrics capture a value pertaining to your systems at a specific point in time—for example, the number of users currently logged in to a web application. Therefore, metrics are usually collected once per second, one per minute, or at another regular interval to monitor a system over time. There are two important categories of metrics in our framework: work metrics and resource metrics. For each system that is part of your software infrastructure, consider which work metrics and resource metrics are reasonably available, and collect them all.

Work metrics

Work metrics indicate the top-level health of your system by measuring its useful output. When considering your work metrics, it’s often helpful to break them down into four subtypes:

  • throughput is the amount of work the system is doing per unit time. Throughput is usually recorded as an absolute number.
  • success metrics represent the percentage of work that was executed successfully.
  • error metrics capture the number of erroneous results, usually expressed as a rate of errors per unit time or normalized by the throughput to yield errors per unit of work. Error metrics are often captured separately from success metrics when there are several potential sources of error, some of which are more serious or actionable than others.
  • performance metrics quantify how efficiently a component is doing its work. The most common performance metric is latency, which represents the time required to complete a unit of work. Latency can be expressed as an average or as a percentile, such as “99% of requests returned within 0.1s”.

Resource Metrics

Most components of your software infrastructure serve as a resource to other systems. Some resources are low-level—for instance, a server’s resources include such physical components as CPU, memory, disks, and network interfaces. But a higher-level component, such as a database or a geolocation microservice, can also be considered a resource if another system requires that component to produce work.

Resource metrics are especially valuable for investigation and diagnosis of problems. For each resource in your system, try to collect metrics that cover four key areas:

  1. utilization is the percentage of time that the resource is busy, or the percentage of the resource’s capacity that is in use.
  2. saturation is a measure of the amount of requested work that the resource cannot yet service, often queued.
  3. errors represent internal errors that may not be observable in the work the resource produces.
  4. availability represents the percentage of time that the resource responded to requests. This metric is only well-defined for resources that can be actively and regularly checked for availability.

    Other metrics

    There are a few other types of metrics that are neither work nor resource metrics, but that nonetheless may come in handy in diagnosing causes of problems. Common examples include counts of cache hits or database locks. When in doubt, capture the data.


    In addition to metrics, which are collected more or less continuously, some monitoring systems can also capture events: discrete, infrequent occurrences that can provide crucial context for understanding what changed in your system’s behavior. Some examples:

    • Changes: Internal code releases, builds, and build failures
    • Alerts: Internally generated alerts or third-party notifications
    • Scaling events: Adding or subtracting hosts

    An event usually carries enough information that it can be interpreted on its own, unlike a single metric data point, which is generally only meaningful in context. Events capture what happened, at a point in time, with optional additional information.Events are sometimes used to generate alerts—someone should be notified of events such as the third example in the table above, which indicates that critical work has failed. But more often they are used to investigate issues and correlate across systems. In general, think of events like metrics—they are valuable data to be collected wherever it is feasible.

Conclusion: Collect them all

  • Instrument everything and collect as many work metrics, resource metrics, and events as you reasonably can.
  • Collect metrics with sufficient granularity to make important spikes and dips visible. The specific granularity depends on the system you are measuring, the cost of measuring and a typical duration between changes in metrics—seconds for memory or CPU metrics, minutes for energy consumption, and so on.
  • To maximize the value of your data, tag metrics and events with several scopes, and retain them at full granularity for at least a year.

Difference between Monitoring and Moderation..



Lots of people are talking about all the new forms of marketing a company can pursue. It’s true, certain traditional marketing has been around for a long time and is still used today, but with the Internet now playing such a huge role in any company’s success, people are coming out with more and more ways to market their products or services. The more we thought about all the different varieties of marketing, the more we realized there are so many different ways to promote something. Here’s a list of marketing terms that we hope you find useful:

Internet Marketing

Internet marketing is any marketing strategy that takes place online. Also referred to as online marketing, it encompasses a variety of marketing forms like video advertisements, search engine marketing and e-mail marketing. It is the opposite of offline marketing, and can also fall under digital marketing. Internet marketing needs a good approach in areas of design, development and advertising. A company with a total web site marketing plan will have more success online than one that has just designed a web site without thinking of how to market their company through it.

Offline Marketing

Offline marketing, the opposite of online marketing, includes all forms of marketing that aren’t done on the Internet. Examples of offline marketing are local advertising in newspapers and on television. In today’s marketing world, companies are finding ways to leverage their offline marketing campaigns with their online ones, making them complement each other.

Outbound Marketing

When you think of marketing, the different forms you come up with are mostly outbound marketing (also called traditional marketing). In fact, the majority of companies today are using different types of outbound marketing to reach their potential customers. Outbound marketing includes any marketing efforts that are taken to introduce a product or service to someone who isn’t looking for that product or service. Some examples are cold calling, sending newsletters, billboards, and banner ads on different web sites.

Inbound Marketing

Inbound marketing focuses on having your company found by customers, as opposed to reaching out to them directly like in outbound marketing. The important thing to remember here is that a person starts out with the want/need to purchase a product or service, and they go out to find it. When they search for that product/service on a search engine, the search engine results page will show inbound marketing results. Instead of using paid advertisements, inbound marketing is the search engine optimization (SEO) part of web marketing.

Newsletter Marketing

Newsletter marketing and email marketing refer to ways of promoting your company through emails. Typically, a firm using newsletter marketing will have a group of contacts that they will send a newsletter containing some interesting information to. The success of newsletter marketing depends on grabbing attention, writing good content and reaching a large number of potential clients.

Article Marketing

Businesses will often write articles related to the industry they are in and distribute them online and offline. These free articles will inform people about an important topic and give the company that wrote it more credibility within the market. The organization can also include their business contact information in the article, allowing them to get new clients.

Trade Show Marketing

Companies that want to reach a large number of potential customers can participate in public or private trade shows. Trade shows and other forms of event marketing are often a large investment to participate in, but trade shows allow companies to demonstrate new products and examine what is going on in the industry.

Search Marketing

Search engine marketing (SEM) is the way in which companies promote their business through paid placement on search engines like Google. Instead of increasing the organic search results that a website has, companies will pay to have their advertisements in the sponsored section of search engines. This is also known as Pay Per Click Advertising or PPC

Direct Marketing

Direct marketing’s main goal is to send a message directly to consumers, without having to use any third party outlets. Examples of direct marketing include mail marketing, telemarketing and direct selling. Direct marketing is often preferable because the results can be easily measured, giving the marketer a better understanding of the success of that campaign.

Niche Marketing

When a product or service is not being readily supplied to a certain portion of a market, a company can focus their efforts on that niche to address a need that isn’t currently being addressed. This targeted marketing is successful because the marketer has identified a need that isn’t being resolved by mainstream providers. Sometimes it is beneficial for a company to focus on a niche instead of trying to compete in a larger market.

Drip marketing

Drip marketing is the act of sending out scheduled targeted emails that are all coordinated to a specific goal of client conversion. The sender uses email marketing software that allows them to setup multiple emails at one time and let them “drip” over time. This sometimes includes phone calls to check in on the clients along the way.

Social Media Marketing

Social network marketing and social media campaigns provide a window to market a product or service on the Internet through different social networks. Companies can use these outlets for their marketing, customer service and sales. The most common and successful means of social media marketing are found on sites like Facebook, Twitter, LinkedIn, YouTube and even company blogs.

Referral Marketing

One of the less strategic types of marketing, referral marketing relies on a company’s customers to refer new customers to that company. Also called word of mouth marketing, this is a more spontaneous way of receiving new business, and can not be solely relied on because results aren’t very predictable. However, word of mouth is still a powerful part of a company’s efforts to bring in new business, especially in the social media community where communication travels freely.

Guerrilla Marketing

With a smaller budget, guerrilla marketing makes a splash by relying on energy, timing and unusual approaches to get the consumer’s attention. The unconventional marketing involved tries to get the most out something small, and make a lasting brand image in the consumer’s mind.

Promotional Marketing

Promotional marketing is a common form of marketing strategy that companies use to motivate a consumer to make a decision and purchase their product. There are a number of ways that businesses will promote a product or service, including holding contests to win a prize, offering coupons for purchasing a product at a discount, and having samples of the product so people can experience it before they purchase.

Affiliate Marketing

Affiliate marketing most likely involves four different groups that contribute to the marketing effort. The Merchant is the company that is producing and selling the product, the Network is the outlet that is used to promote the affiliate link, the Publisher or Affiliate is the person who has the website with the affiliate ad and of course the customer doing the purchasing. Affiliate links are found on all types of websites, and they are used to drive traffic to outside websites.

Viral Marketing

This type of marketing relies on the message of a marketer being spread quickly through various social networks in order to increase brand awareness. The name viral marketing stems from the rapid spread of viruses in general. Typically, a viral marketing campaign will not last as long as other marketing efforts, but if a company can come up with a good idea for viral marketing and reach the right people, it will become highly successful in a short amount of time.

B2B Marketing

Any type of business, whether an organization, individual, government or other institution that markets to other businesses is involved in business to business marketing. Since B2B marketing involves companies trying to sell mass quantities of product to one another, there is a more personal relationship that needs to be established between businesses. If your company sells to other businesses, your marketing efforts will most likely be more direct.

B2C Marketing

Business to consumer marketing campaigns try to reach a category of people that will be likely to purchase their product or service. The marketing efforts the company takes should be more broad than B2B, which focuses on specific companies. B2C marketing can involve different marketing techniques such as door to door marketing, promotion marketing, newspaper marketing, television marketing and radio marketing. In today’s marketing world, B2C Internet marketing is becoming more important to reach consumers.

Mobile Marketing

Along with Internet marketing, mobile marketing is part of the newest groups of marketing activities. Companies have been experimenting with the certain ways to reach consumers through their phones, especially with the rise of Apple’s iphone. Some ways to marketing a product or service through a mobile phone include SMS marketing, in-game marketing, banner marketing on different web pages and location based marketing.

Reverse Marketing

This form of marketing is similar to inbound marketing. The goal of reverse marketing is to market a product in a way that will cause the consumer to seek the firm doing the marketing. Reverse marketing can be conducted through such means as television, print and Internet marketing. If a company has a product that solves a problem in the market, they will have more success using reverse marketing because they will seek out that product.


A form of direct marketing, telemarketing’s focus is on reaching consumers by phone. Most of what we thing of as telemarketing is cold call marketing, which is unpopular and has lead to laws being created against it. However, telemarketing can be effective if the right person is reached on the phone at the right time.

Direct Mail Marketing

Most people receive large quantities of marketing material in the mail, which is considered direct mail marketing. Companies will send paper mail with promotions or other information to a list of addresses, usually in a common geographical area. This form of marketing is also called junk mail by some, because the customers receiving the mail aren’t expecting it and usually don’t want to open it.

Database Marketing

Database marketing is similar to other types of direct marketing, but the focus is more directed towards analyzing data. Companies try to narrow their marketing efforts down to certain groups of people, and they use database marketing to analyze statistics like name, address, or sales history, in order to create the most accurate model possible.

Personalized marketing

The goal of personalized marketing is to create a unique offer for each individual customer. This form of marketing doesn’t work for every company, but certain ones can capitalize on their unique products and customer demographics to market to individuals. With the Internet becoming a more popular place for marketing, companies are finding that personalized marketing is affective in cases when they can track a customer’s specific interests and send them more information for future suggestions.

This list should give you a good idea of different ways companies can market themselves to consumers or other companies, but there are still more types of marketing out there.

Are there any cons of mystery shopping?

Mystery shopping definitely has its place in data collection and market research, but it’s not a perfect study.

  • Low Data Volume. Unlike in-store surveys, online polls, or focus groups, mystery shopping is a one-man job. Because a mystery shopping excursion takes time and money (and because the shopper can only visit one store at a time) data volume can be low. After all, a singular shopper can only gather so much information.
  • Shallow Data. A mystery shopper is his own person, which means he brings his own preferences, habits, and shopping sense to the table. Mystery shopping tells you how one type of customer shops, but not how all types of customer shops. It’s important to compare mystery shopping results with those of your ideal customer profile.
  • Bias and Validity. Mystery shopping is completely subjective, no matter how unbiased the mystery shopper claims to be. Personal bias can skew results enough so that validity can be called into question. ‘
  • Singular Experience. A mystery shopper’s experience in a store only represents one interaction with employees and location. A busy day, short staff, or any number of mishaps can affect one experience, but not truly represent a regular customer visit.

Mystery shopping can offer real data from real people, but take care: bias and missteps can affect results. It’s best to use mystery shopping not as a mass data collection technique, but a method for testing and refining customer experience on a case-by-case basis.

If you want to take care of the aforementioned cons and reap the benefits of the following pros, click here.

Before hiring a shopper or shopper service, brands must be clear on what information and experiences a shopper should be on the lookout for. When done properly and thoroughly, mystery shopping reveals a bevy of benefits.

  • Competitor Analysis. Sending a mystery shopper to evaluate a brand and its competitor side-by-side can yield in-depth analysis on the products, policies, and locations of a nearby retail rival. A mystery shopper can stage similar experiences–returning an item, for instance–to see how both stores stack up. In the realm of market research VS competitive intelligence, this is very valuable.
  • Practice New Policies. When a retail store implements new policies, it needs to see those policies put to practice. Mystery shoppers can mimic natural circumstances to see whether or not employees are adhering to rules and regulations.
  • Employee Evaluation. If the boss is evaluating an employee, he’s bound to be on his best behavior. A mystery shopper ensures a realistic evaluation based on how employees interact with real customers and not just their supervisors.
  • Informal Opinions. Surveys and polls can be helpful to discern customer opinions, but they can be skewed toward the positive. Mystery shoppers can offer informal opinions on everything from point-of-sale to products, displays, customer service, and even the cleanliness of a retail location. Those opinions can be used to better understand customers and make improvements accordingly.

Do you have it in you?

What does it take to be a mystery shopper?

Acting ability:
You need not just to assume an identity, but know your character inside out.

A cool head:
Yes, you’re clandestinely filming the person you’re talking to, but you mustn’t show any outward signs of nervousness.

A good memory:
If you’re operating without a hidden camera, you’ll need to have clear recall of how your mystery shop went.

Knowledge of the subject:
If you’re registering to buy a shared-equity or FirstBuy property, you need to have a grasp of how the finances work, otherwise you’ll get rumbled.

It’s not just property assignments you can be given. The clients for whom Mystery Shoppers Ltd work include funeral directors, motorway toll roads, department stores and car-repair garages.

Mystery Shopping Decoded

Tips vital to making money in this somewhat misunderstood profession.

1. Mystery shopping can be tough to get into.

If you’re OK with starting out doing $8 gigs for ordering a fast food meal, then chances are great that you can begin right away. The higher paying shops, however, are usually reserved for those with experience. If you want to earn a reputation for being a dependable shopper, I suggest visiting the a suitable website and getting at least a silver status certification. Yes, it costs money. (Think of it as a “move to the head of the line” pass for most mystery shopping jobs.) Many reputable companies will only schedule certified shoppers for their jobs.

2. You can’t earn money if you don’t spend it.

Guess what? Mystery shopping requires you to shop (in most cases). Since they don’t mail you cash up front, it is your responsibility to be able to cover your purchases for the shop. It also takes between 30 and 90 days to get reimbursed. If you can’t spare this money, this may not be the job for you.

3. A successful shop requires an eye for detail.

I loved doing the fine dining shops. The problem was, I had a difficult time remembering all the details I needed to complete the shop. I had to covertly keep tabs on the names of every person I came in contact with, what they were wearing, what they said, what my food tasted like, etc. Needless to say it was work! If you are looking for mystery shopping to be your free meal ticket, understand that there will not be much time for relaxing. While it is true that some shops require little work, others require much, much more.

4. Payment depends on your performance.

Unlike a typical 9-5 job, you are not guaranteed payment unless your shop has been performed satisfactorily. If you forget the names of your wait staff, don’t leave the right amount of tip, or accidentally reveal your shopper status, you are putting your reimbursement in danger. I have never had a shopping company not pay me, but I have also been very diligent about doing everything perfectly. If you don’t take it seriously, you may not be paid — and you’ll be out whatever cash you put into your shopping experience.

5. There are other costs involved.

In addition to the cost of your shopping (which is usually reimbursed partially or in full), there are other costs. Gas to drive to the shop, the cost (if any) to put an item on your credit card until reimbursement, or the cost of a tip (which is often not covered) are just a few expenses that may come up during a typical shop. Obviously, the best strategy is to shop close, only take shops that reimburse in full (and with an extra shopper’s fee, if possible), and turn in your reports on time.

6. You are responsible for your own taxes.

As a mystery shopper, you are considered an independent contractor. While it is unlikely that you will earn over $600 a year for any one company, you will still be responsible for reporting that income on your tax returns. You can count it as self-employment, deducting expenses as needed, so keep track of the cost of your new mystery shopping job.

7. Some mystery shopping isn’t shopping or a mystery.

Many shopping companies have begun scheduling work for companies that aren’t even related to mystery shopping. Audits, merchandising, and other tasks (including headstone cleaning) often come up on the mystery shopping job boards from time to time. If you don’t have an interest in these types of jobs, don’t feel obliged to take them. They can be a good source of income for you, however.

8. A reputable mystery shopping company will never ask you for any kind of fee.

I’ll say this again: You should not have to ever pay for the “privilege” to shop. You are performing a service, and should get paid. Any fee that is guaranteed to get you a list of jobs is bogus. It costs nothing.