The Exchange team is announcing today the availability of our most recent quarterly servicing update to Exchange Server 2013. Cumulative Update 5 for Exchange Server 2013 and updated UM Language Packs are now available on the Microsoft Download Center. Cumulative Update 5 represents the continuation of our Exchange Server 2013 servicing and builds upon Exchange Server 2013 Service Pack 1. The release includes fixes for customer reported issues, minor product enhancements and previously released security bulletins. A complete list of customer reported issues resolved in Exchange Server 2013 Cumulative Update 5 can be found in Knowledge Base Article KB2936880. Customers running any previous release of Exchange Server 2013 can move directly to Cumulative Update 5 today. Customers deploying Exchange Server 2013 for the first time may skip previous releases and start their deployment with Cumulative Update 5 as well.

Note: Some article links may not be available at the time of this post's publication. Updated Exchange 2013 documentation, including Release Notes, will be available on TechNet soon.

We would like to call your attention to a couple of items in particular about the Cumulative Update 5 release:

• Based upon customer feedback, we have introduced improvements to OAB management for distributed environments. You can read more about this in a post by Ross Smith IV on the Exchange Team blog. Customers who have deployed Multiple OAB Generation Mailboxes are advised to read this post to help avoid unnecessary OAB downloads.
• Cumulative Update 5 includes a Managed Availability probe configuration that is frequently restarting the Microsoft Exchange Shared Cache Service in some environments. The service is being added to provide future performance improvements and is not used in Cumulative Update 5. More information is available in KB2971467.

For the latest information and product announcements please read What’s New in Exchange Server 2013, Release Notes and product documentation available on TechNet.

Cumulative Update 5 includes Exchange related updates to Active Directory schema and configuration. For information on extending schema and configuring the active directory please review the appropriate TechNet documentation. Also, to prevent installation issues you should ensure that the Windows PowerShell Script Execution Policy is set to “Unrestricted” on the server being upgraded or installed. To verify the policy settings, run the Get-ExecutionPolicy cmdlet from PowerShell on the machine being upgraded. If the policies are NOT set to Unrestricted you should use the resolution steps in KB981474 to adjust the settings.

Reminder: Customers in hybrid deployments where Exchange is deployed in-house and in the cloud, or who are using Exchange Online Archiving with their in-house Exchange deployment are required to maintain currency on Cumulative Update releases.

The Exchange Team

This article continues the analysis I started in my previous article, DAG: beyond the “A”.

We all understand that a good technology solution must have high levels of availability, and that simplicity and redundancy are the two major factors that drive solution availability. More specifically:

• The simpler the solution (the fewer independent critical components it has), the higher the availability of the solution;
• The more redundant the solution (the more of multiple, identical components that duplicate each other and provide redundant functionality), the higher the availability of solution.

My previous article provides mathematical formulas that allow you to calculate planned availability levels for your specific designs. However, this analysis was performed from a standpoint of a single datacenter (site). Recently the question was asked: how does bringing site resilience to an Exchange design affect the solution's overall level of availability? How much, if any, will we increase overall solution availability if we deploy Exchange in a site resilient configuration? Is it worth it?

Availability of a Single Datacenter Solution

Let us reiterate some of the important points for a single site/datacenter solution first. Within a datacenter, there are multiple critical components of a solution, and the availability of an entire solution can be analyzed using the principles described in DAG: beyond the “A”, based on the individual availability and redundancy levels of the solution components.

Most interestingly, availability depends on the number of redundant database copies deployed. If the availability of a single database copy is A = 1–P (this includes the database copy, and the server and disk that are hosting it), then the availability of a set of N database copies will be A(N) = 1­–P^N = 1–(1–A)^N. The more copies, the higher the availability; the fewer copies, the lower the availability. The graph below illustrates this formula showing the dependency of A(N) on N:

Figure 1: Availability dependence on the number of redundant copies

Note: All plots in this article were built using the Wolfram Alpha online mathematical computation engine.

For example, if A = 90% (value selected on the graph above), and N=4, then A(4) = 99.99%.

However, the full solution consists not just of the redundant database copies but of many other critical components, as well: Active Directory, DNS, load balancing, network, power, etc. We can assume that the availability of these components remains the same regardless of how many database copies are deployed. Let’s say the overall availability of all of these components taken together in a given datacenter is A_infra. Then, the overall availability of a solution that has N database copies deployed in a single datacenter, is A₁(N)= A_infrax A(N).

For example, if A_infra = 99.9%, A = 90%, and N=4, then A₁(4)= 99.89%.

Adding Site Resilience

So we figured that availability of a single datacenter solution is A₁ (for example, A₁=99.9%=0.999). Correspondingly, probability of a datacenter failure is P₁=1–A₁ (in this example, P₁=0.1%=0.001).

Let’s assume that a second site/datacenter has availability of A₂ (it could be the same value as A₁ or it could be different – it depends on the site configuration). Correspondingly, its probability of failure is P₂=1– A₂.

Site resilience means that if the solution fails in the first datacenter, there is still a second datacenter that can take over and continue servicing users. Therefore, with site resilience the solution will fail only when *both* datacenters fail.

If both datacenters are fully independent and don’t share any failure domains (for example, they don’t depend on the same power source or network core switch), then the probability of a failure of both datacenters is P= P₁xP₂. Correspondingly, the availability of the solution that involves site resilience based on two datacenters is A = 1–P = 1–(1–A₁)x(1–A₂).

Because values of both P₁ and P₂ are very small, the availability of a site resilient solution effectively sums the “number of nines” for both datacenters. In other words, if DC1 has 3 nines availability (99.9%), and DC2 has 2 nines availability (99%), the combined site resilient solution will have 5 nines availability (99.999%).

This is actually a very interesting result. For illustration, let us use datacenter tier definitions adopted by ANSI/TIA (Standard TIA-942) and the Uptime Institute, with the availability values for four datacenter tiers defined as follows:

We can see that if we deploy two relatively inexpensive Tier 2 datacenters, the resulting availability of the solution will be higher than if we deploy one very expensive Tier 4 datacenter:

Of course, this logic applies not only to datacenter considerations but also to any solution that involves redundant components. Instead of deploying an expensive single component (e.g., a disk, a server, a SAN, a switch) with a very high level of availability, it might be cheaper to deploy two or three less expensive components with properly implemented redundancy, and it will actually result in better availability. This is one of the fundamental reasons why we recommend using redundant commodity servers and storage in the Exchange Preferred Architecturemodel.

Practical Impact of Site Resilience

The advantage of having two site resilient datacenters instead of a single datacenter is obvious if we assume that site resilient solutions are based on the same single datacenter design implemented in each of the two redundant datacenters. For example, if we compare one site with 2 database copies and two sites with 2 database copies in each, obviously the second solution has much higher availability, not so much because of site resilience but simply because now we have more total copies – we moved from 2 total copies to 4.

But this is not a fair comparison. What is the effect of the site resilience configuration itself? What if we compare the single datacenter solution and the site resilient solution when they have the same number of copies? For example, single datacenter solution with 4 database copies and a site resilient solution with two sites with 2 database copies in each site (so that both solutions have 4 total database copies). Here the calculation becomes more complex.

Using the results from above, let’s say the availability of a solution with the single site and M database copies is A₁(M) (for example, A₁(4)=99.9%=0.999). Obviously, availability of the same solution but with fewer database copies will be lower, (for example, A₁(2)=90%=0.9).

Let’s assume similar logic for the second site: let it have N copies and a corresponding availability of A₂(N).

Now we need to compare the following values:

• Availability of a single site solution with M+N copies: A_S = A₁(M+N)
• Availability of a site resilient solution with M copies in the 1^st site and N copies in the 2^nd site:
  A_SR = 1­–(1–A₁(M))x(1–A₂(N))

These values are not very easy to calculate, so let us assume for simplicity that both datacenters are equivalent (A₁ = A₂) and both have equal number of copies (M=N). Then we have:

A_S = A₁(2N)

A_SR = 1­–(1– A₁(N))²

We know that A₁ = A_infra x A(N), and that A(N) = 1­–P^N = 1–(1–A)^N. Since we consider datacenters equivalent, we can assume that A_infrais the same for both datacenters. This gives us:

A_S = A_infra x (1–(1–A)^2N)

A_SR = 1­–(1– A_infra x (1–(1–A)^N))²

These values depend on three variables: A_infra, A, and N.

To compare these values, let us fix two of the variables and see how the result depends on the third one.

One comparison is to see how the values change depending on A if A_infra and N are fixed. For example, let A_infra= 99% = 0.99, and N=2:

Figure 2: Availability dependence on number of redundant copies for the single site and site resilient scenarios

The blue line (bottom curved line) represents the single datacenter solution, and the purple line (top curved line) represents the site resilient solution. We can see that site resilient solution always provides better availability, and the difference is steady even if the availability of an individual database copy approaches 1. This is because the availability of other critical components (A_infra) is not perfect. The better A_infra(the closer it is to 1), the smaller the difference between the two solutions.

To perform another comparison and confirm the last conclusion, let us see how availability changes depending on A_infraif A and N are fixed. For example, let A=0.9 and N=2:

Figure 3: Availability dependence on the datacenter infrastructure availability for the single site and site resilient scenarios

Again, we can see that the site resilient solution provides better availability but the difference between the two availability results is proportional to 1–A_infra and so it vanishes when A_infra–>1, which confirms the conclusion made earlier.

In other words, if your single datacenter has a perfect 100% availability, then site resilient solution is not needed. Now isn’t that obvious without any calculations?

The following table illustrates these results:

You can leverage this simple Excel spreadsheet that allows you to play with the numbers representing A_infra, A, and N (they are formatted in red), and see for yourself how it affects resulting availability values.

Summary

Deploying a site resilient design increases availability of a solution, but the benefit of site resilience diminishes if a single datacenter solution has high level of availability by itself.

Using the formulas above, you can calculate exact availability levels for your specific scenarios if you use proper input values.

Note: To avoid confusion, everywhere above we are talking about planned availability. This purely theoretical value demonstrates what can be expectedof a given solution. On comparison, the actually observed availability is a statistical result; in actual operations, you might observe better or worse availability values, but the averages over the long period of monitoring should be close to the theoretical values.

Acknowledgement: Author is grateful to Ramon Infante, Director of Worldwide Messaging Community at Microsoft, and Jeffrey Rosen, Solution Architect and US Messaging Community Lead, for helpful and stimulating discussions.

Boris Lokhvitsky
Delivery Architect
Microsoft Consulting Services

It has been about two months since I was in Austin at my first Microsoft Exchange Conference. I must say Austin left quite an impression on me, partly because of the wonderful people and great music, but mostly because of the immensely passionate public folder customers I got a chance to meet. Public folders are one of the oldest living artifacts in Exchange Server and MEC was a reconfirmation that they continue to serve some critical and unique collaboration scenarios for our customers. MEC allowed me to meet with some of our largest and most complex public folder customers to discuss in great detail how some companies’ entire business processes rely on public folders.

At MEC, customers asked us for higher scale and more functionality in Exchange modern public folders. The 3 top asks in the order of priority for the public folder team included:

1. Scale improvements
2. OWA support for calendar and contacts
3. Better public folder reporting tools

What's new on scale?

As discussed at MEC, scaling up modern public folders is a priority for us. We have already made significant strides towards improving public folder scale. We have removed the design limitation which was the root cause for the hard limit of 10,000 total folders - this opens up new doors for scale. Synchronization of public folder hierarchy was a major component responsible for the 10,000 folder limitation. Full hierarchy sync has been improved to the point where it can scale linearly as folder count grows within the environment. There have also been storage optimizations which will provide better performance in operations on public folders in run state.

What does this mean for public folder counts?

The good news is that with these recent changes you can expect to see higher limits on public folders rolled out in the next few months. Office 365 customers should see the total folder limit increase to 100,000 by early July. For our large on premises customers, we are targeting to increase this limit further with the CU6 release. As we get closer to the CU6 date we will publish an update on how far we were able to push the limit for on premises customers.

This is just the first round in the public folder scale improvements we are working to deliver. As we move forward with these improvements, our goal is to scale to at least 1 million folders. We are also looking into making improvements to migration as well as store more bytes in more public folder mailboxes with more users accessing them.

Why CU6?

Scale updates is complex business. Investigating and working through fixes on scale required a significant lead time and these updates didn’t fit into the CU5 timeline as work from multiple teams with features dependent on each other were involved. So we plan to roll in the above mentioned scale improvements in CU6 for our on-premises customers. Office 365 customers however will see these updates as soon as the code containing all of the necessary public folder scale improvements rolls out into service. This service update is targeted for early July, but as with all office 365 updates some tenants will see it sooner than others until the service update has been deployed world-wide. We will be publishing another blog around the CU6 timeframe to share news about the next round of public folder scale updates.

What about the other asks?

We expect calendar and contact public folders support in OWA to follow the scale updates. We do not have exact timelines on these yet. Following calendar & contact folder support in OWA we will be working to deliver additional management tooling to assist in your management of public folders. Stay tuned for updates.

At Exchange we live to solve tough problems. Public folders were re-architected in Exchange 2013 to take advantage of the well proven mailbox design and deliver a solution for customers moving to Office 365 with public folders. We will continue building towards this promise as we proceed to meet customer requirements. Thanks to the customers who provided input at MEC.  We will keep you updated with further details in advance on the Exchange Blog.

Kanika Ramji
Program Manager, Exchange Public Folders

In the last week there have been two big improvements related to CalCheck and OffCAT that will make life a little easier when troubleshooting ‘calendar’ issues.

1. Relevant solutions for all CalCheck issues added to the following article:

2678030  Information about the Calendar Checking Tool for Outlook (CalCheck)

The detailed solutions added under each type of error/warning flagged by CalCheck will help you resolve/fix meetings more quickly.

Also, there is an HTML bookmark on each bullet item in the article so OffCAT can point you directly to the solution instead of requiring you to find the solution in the above article.

2. Addition of a new node in OffCAT under Configuration Details called Problem meeting details.

• Under this node you will find each meeting in your calendar that has been flagged as a Warning or Error by CalCheck.
• When you expand the meeting in the tree you will see:
  • The error/warning text from CalCheck (describing the exact problem with the meeting)
  • A ‘Click Here for the solution’ hyperlink

This link will take you directly to the bookmarked section of article 2678030 so you can go right to the section of the article that applies to the error/warning flagged by CalCheck.

How this looks in OffCAT

To see this new functionality in OffCAT, just search for ‘calendar checking’ in Configuration Details to see this new node.

The link for the error shown in the above figure takes you to the ‘No dispidRecurring property’ section of the article under Item-level checks.

And, the solution for this problem is found directly underneath.

Thanks,

The OffCAT team

While we do not post about purely development subjects very often, I wanted to point you to a pretty cool post on the Exchange dev blog:

Zapier’s Office 365 API Journey

It may come as a surprise to some just how seamless it is to jump into leveraging Office 365 APIs. If this interests you, enjoy!

Nino Bilic

Quick note to let you know we've announced a new unified technology event for enterprises. If you've attended Microsoft Exchange Conference (MEC), SharePoint Conference, Lync Conference or Project Conference, this event is for you! Head over to Office Blogs for more.

Bharat Suneja

I wanted to write a series of blog posts talking about email spam in Office 365. While majority of spam mail is blocked by the Office 365 mail security gateways, there are no perfect systems that will block 100% of spam all the time, some can still get through. In case that we do experience spam mail, we can use several tools and configuration options that are available for us in Office 365 to deal with it and improve effectiveness.

In this series, we will quickly review different types of spam mail. Then we will present different tools that we can use for fighting spam mail in an Office 365 environment and try to “match” the “spam tool” for the task based on the type of the spam.

Also please note that while we are approaching this from Office 365 viewpoint, many of the procedures listed here apply to both on-premises and hybrid deployments.

Introduction

One of the advantages of using Office 365 is that transparently, behind the scenes, we implement EOP – Exchange Online Protection (the former mail security infrastructure was implemented by FOPE services).

The Exchange Online Protection infrastructure serves as mail gateways, which are responsible for the “Hygiene” of incoming and outgoing mail flow. The purpose of this mail gateway’s is to filter any malware, virus or spam that might be included in the mail flow that comes from external sources to Office 365 recipients (incoming mail flow) and also mail that is sent from Office 365 recipients to external sources. A bit over-simplified but think of it like this:

What should I do when EOP is not working as desired?

EOP aims to provide the best possible protection, but from time to time Office 365 subscribers can experience spam mail that gets into their mailbox.

Before going further into this, let’s not forget that there is no “perfect solution” that will block 100% of spam mail because “spam solutions\gateways”, will always need to face issues of:

• False Positive - a scenario in which the defending systems recognize legitimate mail is bad\spam mail and blocks the mail. 
• False Negative - a scenario in which the defending system doesn’t recognize bad\spam mail and the mail reaches recipients mailbox.

Certainly any hygiene solution, even a cloud-based one, will have times when a few messages originating from a creative spammer sneak through before it is recognized as a threat. The advantage that a cloud-based solution offers is that it is set up to recognize those threats quickly, partially due to the quantity of email that it processes.

Additionally, different users will always have slightly different expectations. It is therefore challenging to have a default configuration setting that is perfect for different business customers, each with unique requirements. One person’s spam email can be another person’s legitimate business email. EOP defaults tend to be slightly less strict rather than risk a false positive. If these defaults are not adequate for your organization, EOP offers great flexibility in allowing customization of anti-spam settings.

This series of blog posts will help you understand what to do in either situation.

Spam mail - Troubleshooting process and classification

To create a clear path of the troubleshooting process, we will need to implement the workflow similar to the one in the following diagram:

Step 1 - Get information about the character of the spam mail.

The most basic step is to get essential information about the spam message. Determine if the mail message is truly a spam message and if so, try to recognize the type of spam. Based on this information, choose the right “tools” for mitigating it (we will cover more of those in future posts).

Questions to answer

Here is a list of questions that could help gather required information:

• Q: Is the mail considered as spam mail or just standard advertisement mail from a well-known\familiar company?
• Q: Is the spam mail sent from a specific sender email address?
• Q: Is the spam mail sent from a specific domain?
• Q: Does the spam mail include specific keywords in the mail subject\body?
• Q: Does the spam mail include specific URLs in the mail body that redirect the recipient to another location?
• Q: Does the spam mail include characters of non-English language?
• Q: Is the spam mail from a specific geographical location?
• Q: Is the spam mail directed to a specific user or distribution list in the organization?

General characteristics:

• Q: Is the spam mail sent on a specific schedule (specific hour or date)?
• Q: What is the percentage of organization users who get the spam mail?
• Q: What is the “amount” of the spam mail (single mail item, tens or hundreds of spam mails)?
• Q: How long has the spam mail been received (days/hours)?
• Q: When was the last spam mail received?

Step 2 – Report\Block spam mail

When we deal with spam mail, we need to try to block the spam mail by using the available option from the “Server side” (Exchange online and EOP) and the “Client side” (Outlook). The process of blocking the spam mail could be implemented as a combined operation of using tools for filtering spam mail and other tools for reporting (sending a sample of the spam mail) to the Microsoft team that manages the EOP infrastructure.

Dealing with spam mail - Client side

1. Microsoft Junk E-mail Reporting Add-in

The Microsoft Junk E-mail Reporting Add-in, is a very useful Outlook add-in that enables each of the users to report the offending message to Microsoft.

By selecting the mail item and then choosing the option of “Report Junk," the mail item will automatically be sent to the Microsoft mail security team for further analysis and investigation to help to improve the effectiveness of our junk e-mail filtering technologies.

Using the Microsoft Junk E-mail Reporting Add-in

• Download and Install the Microsoft Junk E-mail Reporting Add-in.
• Report email as spam

In Outlook 2010\2013, the Microsoft Junk E-mail Reporting Add-in is implemented by additional menu option named: Report junk that is added to the “Junk” section to be able to report an email as spam. To “mark” mail item as Junk use the following procedure: 

• Choose the mail items you would like to report
• On the Home Tab choose the small black arrow of the Junk option.
• Choose the option Report Junk

A warning message appears and informs the user that the mail item will be reported as spam. Choose the “Yes” option.

When we choose the “yes” option, the following events will occur:

• The mail items that were reported as spam, will be sent to the Junk Email folder.
• A copy of mail items will be sent to abuse@messaging.microsoft.com as attachments, as can be seen in the sent items folder
• An acknowledgement email will be sent back to the recipient.

In Outlook 2007, the option to “report junk” will be added on the top menu option.

2. Outlook Junk option - block sender

Another option that is available for us from “client side” is the Outlook junk component and the option of “block sender” (Add a sender to the Blocked Senders list).

This option is most suitable in a scenario that the spam mail is delivered from a specific recipient email address. In reality, many times the “spammers” mange to send the spam mail by using a different source recipient email address, so the option to “block sender” will not help us in such scenarios.

Add a sender to the Blocked Senders list

In case that you want to block the sender who sends spam mail, we can use the junk menu for blocking this recipient.

• Choose the required mail items,
• In the Home Tab chooses the small black arrow of the Junk option.
• Choose the option Block sender

Additional reading:

• How to obtain the latest Outlook Junk E-mail Filter update
• Overview of the Junk Email Filter

3. Unsubscribe from a mailing list

In case that the user reports “spam mail” and when checking the mail item, we see that the sender is not considered as “spammer” (mail is just a standard advertising email that is sent to a distribution list that the user is on), most of the time the mail will include an option that enables the user to unsubscribe from the mailing list.  So, before we start to use the “heavy artillery," please check if the option of “unsubscribe” exists and unsubscribe from the mailing list.

4. Educate users: How to avoid spam

Educating users to avoid spam belongs to a “proactive” section in which we are trying to avoid a scenario that could lead to spam mail. 

By providing our users instructions and guidance about behavior they should avoid, we can prevent or significantly reduce in advance the occurrence of “spam events."

You can read more information about this subject by using the following link:

10 tips on how to help reduce spam

That is all for today – part 2 (starting to talk about server side solutions) to follow soon!

Mohd Imran Shaikh

An important update is now available to resolve issues customers are currently experiencing when using the Hybrid Configuration Wizard (HCW) to create a new or manage an existing hybrid deployment with Microsoft Exchange Server 2013.

If you currently have an Exchange 2013-based hybrid deployment configured, you will not notice any issues unless you rerun the HCW as part of updating or managing your existing hybrid features. Unless you need to reconfigure your hybrid deployment, you can simply wait for the next update of Exchange Server 2013 (Cumulative Update 6) to correct this issue with the HCW.

For Exchange 2013 organizations creating new or managing an existing hybrid configuration with the HCW, the following HCW error message indicates you are experiencing the issue this update addresses:

Subtask CheckPrereqs execution failed: Check Tenant Prerequisites Deserialization fails due to one SerializationException: Microsoft.Exchange.Compliance.Serialization.Formatters.BlockedTypeException: The type to be (de)serialized is not allowed: Microsoft.Exchange.Data.Directory.DirectoryBackendType

If you experience this issue, contact Microsoft support to obtain the fix as documented in KB2988229. This fix requires Exchange Server 2013 Service Pack 1 (SP1) or Cumulative Update 5 (CU5).

Once the Interim Update (IU) is applied, customers can successfully run the HCW and complete configuring a hybrid deployment with Office 365.

Brian Shiers
Technical Product Manager

FAQ

Q: I’ve already configured a hybrid deployment with Exchange Server 2013 and I don’t need to make any changes to my hybrid configuration or features, do I need to apply this update?

A: No, you can wait for the fix to be delivered in the next Exchange Server 2013 update as long as you don’t have the need to run the Hybrid Configuration Wizard.

Q. I may need to make updates to my Exchange 2013-based hybrid deployment before the next Exchange Server 2013 update, what are my options?

A. If you need to update your hybrid deployment features before the next Exchange Server 2013 update, you’ll need to install the IU to fix the issues in the HCW. Attempting to manually configure a new or update an existing hybrid deployment without HCW can result in unsupported hybrid deployment states.

Q: Are customers who use Exchange Server 2010 impacted by this update?

A: No, this only applies to customers using Exchange Server 2013 to configure a hybrid deployment with Office 365.

Q: If we apply the update specific for SP1 or CU5 do I have to do anything special to update to CU6 or later in the future?

A: The interim update does NOT need to be uninstalled. We allow later CU’s to install over Interim Updates and Security Updates directly as of CU3.

Probes are one of the three critical parts of the Managed Availability framework (monitors and responders are the other two). As I wrote previously, monitors are the central components, and you can query monitors to find an up-to-the-minute view of your users’ experience. Probes are how monitors obtain accurate information about that experience.

There are three major categories of probes: recurrent probes, notifications, and checks.

Recurrent Probes

The most common probes are recurrent probes. Each probe runs every few minutes and checks some aspect of service health. They may transmit an e-mail to a monitoring mailbox using Exchange ActiveSync, connect to an RPC endpoint, or establish CAS-to-Mailbox server connectivity. All of these probes are defined in the Microsoft.Exchange.ActiveMonitoring\ProbeDefinition event log channel each time the Exchange Health Manager service is started. The most interesting properties for these events are:

• Name: The name of the Probe. This will begin with the SampleMask of the Probe’s Monitor.
• TypeName:The code object type of the probe that contains the probe’s logic.
• ServiceName: The name of the Health Set for this Probe.
• TargetResource: The object this Probe is validating. This is appended to the Name of the Probe when it is executed to become a Probe Result ResultName
• RecurrenceIntervalSeconds: How often this Probe executes.
• TimeoutSeconds: How long this Probe should wait before failing.

On a typical Exchange 2013 multi-role server, there are hundreds of these probes defined. Many probes are per-database, so this number will increase quickly as you add databases. In most cases, the logic in these probes is defined in code, and not directly discoverable. However, there are two probe types that are common enough to describe in detail, based on the TypeName of the probe:

• Microsoft.Exchange.Monitoring.ActiveMonitoring.ServiceStatus.Probes.GenericServiceProbe: Determines whether the service specified by TargetResource is running.
• Microsoft.Exchange.Monitoring.ActiveMonitoring.ServiceStatus.Probes.EventLogProbe: Logs an error result if the event specified by ExtensionAttributes.RedEventIds has occurred in the ExtensionAttributes.LogName. Success results are logged if the ExtensionAttributes.GreenEventIds is logged. These probes will not work if you override them to watch for a different event.

The basics of a recurrent probe are as follows: start every RecurrenceIntervalSeconds and check (or probe) some aspect of component health. If the component is healthy, the probe passes and writes an informational event to the Microsoft.Exchange.ActiveMonitoring\ProbeResult channel with a ResultType of 3. If the check fails or times out, the probe fails and writes an error event to the same channel. A ResultType of 4 means the check failed and a ResultType of 1 means that it timed out. Many probes will re-run if they timeout, up to the MaxRetryAttempts property.

The ProbeResult channel gets very busy with hundreds of probes running every few minutes and logging an event, so there can be a real impact on the performance of your Exchange server if you perform expensive queries against this event channel in a production environment.

Notifications

Notifications are probes that are not run by the health manager framework, but by some other service on the server. These services perform their own monitoring, and then feed data into the Managed Availability framework by directly writing probe results. You will not see these probes in the ProbeDefinition channel, as this channel only describes probes that are run within the Managed Availability framework.

For example, the ServerOneCopyMonitor Monitor is triggered by Probe results written by the MSExchangeDagMgmt service. This service performs its own monitoring, determines whether there is a problem, and logs a probe result. Most Notification probes have the capability to log both a red event that turns the Monitor Unhealthy and a green event that make the Monitor healthy once more.

Checks

Checks are probes that only log events when a performance counter passes above or below a defined threshold. They are really a special type of Notification probe, as there is a service monitoring the performance counters on the server and logging events to the ProbeResult channel when the configured threshold is met.

To find the counter and threshold that is considered unhealthy, you can look at Monitor Definitions with a Type property of:

· Microsoft.Office.Datacenter.ActiveMonitoring.OverallConsecutiveSampleValueAboveThresholdMonitor or

· Microsoft.Office.Datacenter.ActiveMonitoring.OverallConsecutiveSampleValueBelowThresholdMonitor

This means that the probe the Monitor watches is a Check probe.

How this works with Monitors

From the Monitor’s perspective, all three probe types are the same as they each log to the ProbeResult channel. Every Monitor has a SampleMask property in its definition. As the Monitor executes, it looks for events in the ProbeResult channel that have a ResultName that matches the Monitor’s SampleMask. These events could be from recurrent probes, notifications, or checks. If the Monitor’s thresholds are reached or exceeded, it becomes Unhealthy.

It is worth noting that a single probe failure does not necessarily indicate that something is wrong with the server. It is the design of Monitors to correctly identify when there is a real problem that needs fixing versus a transient issue that resolves itself or was anomalous. This is why many Monitors have thresholds of multiple probe failures before becoming Unhealthy. Even many of these problems can be fixed automatically by Responders, so the best place to look for problems that require manual intervention is in the Microsoft.Exchange.ManagedAvailability\Monitoring crimson channel. These events sometimes also include the most recent probe error message (if the developers of that Health Set view it as relevant when they get paged with that event’s text in Office 365).

There are more details on how Monitors work, and how they can be overridden to use different thresholds in the Managed Availability Monitors article.

Abram Jackson
Program Manager, Exchange Server

In the last related blog post we gave some introduction about Exchange Online Protection (EOP), what needs to be done when EOP is not working as desired and spam email troubleshooting process and classification. In this blog we will be moving further and discussing some more advanced option to stop spam emails.

1. IP Block list

The “IP block list” option enables us to block email messages that came from a specific mail server (specific IP).

EOP - using the the IP Block list

• Login to Office 365 portal, Exchange admin center
• On the left-side menu bar, choose the Protection menu
• On the top menu options, choose the connection filter menu
• Choose the Default connection filter policy
• In the window that appears, choose the option: connection filtering menu.
• In the section: IP block list, Choose the plus icon to add the IP address of the Mail server that sent the spam

Additional reading

• Configure the Connection Filter Policy

2. International spam

The “International spam” is an interesting option that enables us to block or identify mail as “spam” based on the classification of Geographical location or Language.

Note: We need to be cautious when using this option because we can very easily get into the scenario in which legitimate mail is identified as “bad\spam” mail and be blocked.

Using the International spam option

• Login to Office 365 portal, Exchange admin center.
• On the left side menus, choose the protection menu
• On the top menu options, choose the content filter menu
• Choose the Default connection filter policy
• In the window that appears choose the option: international spam menu.

We can use one (or both) of the following options:

Blocking mail written in the specific language

• Choose Filter email messages written in the following languages
• Click on the Plus icon and choose the specific languages that you want to block

Blocking mail by Geographical location

• Choose Filter email messages sent from the following countries or regions
• Click on the Plus icon and choose the specific regions that you want to block

3. Content filter advanced options

Before we begin with instruction of how to use EOP advanced option for spam mail, let’s explore additional classifications of spam mail types and the tools we can use. Using a high level classification, we can define 3 “families” of spam mail types:

• Advertisement mail - negative effect of such mail could be considered as “annoying." No real damage is caused to users besides the fact that the user is troubled by the content of the mail (suggestions to buy different medications, enlarge specific body parts and so on). This type of spam mail is automatically blocked (most of the time) by the Office 365 mail gateways. In case that some Advertisement spam mail manages to “sneak in" we can use a solution such as “rules” for blocking this type of spam mail. 
• Mail with malicious content - this type of spam mail is closer to the definition of “virus” because, the target of the spammer is to cause the destination recipient to click or accept some suggestion that could lead the user to many kinds of attacks such as fraud, phishing and so on.
• “Other spam mail” - in this group, we have other spam mail types that don’t belong to the former families. As an example, we can mention spam mail described as NDR backscatter.

Content Filter - Advanced options

Using Content Filter - Advanced options

• Login to Office 365 portal, Exchange admin center.
• On the left side menus, choose the protection menu
• On the top menu options, choose the content filter menu
• Choose the Default connection filter policy
• In the window that appears choose the option: advanced options menu.

As you can see there are many possible options that we can select. The options are divided into 2 categories: Increase spam Score and, Mark as spam.

To be able to demonstrate options available in the Content Filter - Advanced options let describe two scenarios:

• Scenario 1: Blocking spam mail with malicious content
• Scenario 2: Blocking spam mail classified as NDR backscatter

Scenario 1: Blocking spam mail with malicious content

Over the last month, users were complaining about spam mail that contains malicious content. When users open the mail item, they are automatically redirected to a web site, and once there are invited to download an executable file. To be able to block this spam mail item, we would activate three additional filters: mark as spam if the mail item is or contains:

Empty messages

JavaScript or VBScript in HTML

Frame or IFrame tags in HTML

By default, each of the security filter status is: Off. When we click on the “option arrow," we can see that we can choose the options: “Off," “On” or “Test." In case that we choose the option “On," each mail that contains content that is not allowed by one of the security filters that was selected (such as JavaScript or VBScript in HTML) will be marked as spam.

In case that we just want to test the “new security filter” we can choose the option “Test." In the following screenshot, we can see that we can choose one of the following three options:

• None
• Add the default test X-header text
• Send a BCC message to this address  (Note: This address should have a separate mailbox that is just for testing the security filters.)

Scenario 2: Blocking spam mail classified as NDR backscatter

NDR backscatter is a special kind of spam because the “mechanism” that’s used by the spammer is different from the “Standard spam mail." NDR backscatter is when spammer forges the user’s email address and sends email on their behalf to other recipients. If the “destination mail system” recognizes the mail as a spam or if the mail is sent to non-existing users, the “destination mail system” creates an NDR message that is sent to the organization recipient (the user whose email address was used by the spammer).

Generally speaking, Office 365 security gateway servers are configured to block this kind of spam mails, but in case that the spam mail manages to “sneak” through, we can add the following filter using the Content Filter - Advanced options.

Using Content Filter - Advanced options - NDR backscatter

• Login to Office 365 portal, Exchange admin center.
• On the left side menus, choose the protection menu
• On the top menu options, choose the content filter menu
• Choose the Default connection filter policy
• In the window that appears choose the option: advanced option menu.
• Choose the option: NDR backscatter, and turn on the security filter

That is all for this time. Until we meet again,

Mohd Imran Shaikh

When customers receive an email with a suspected virus, they often ask “What do I do now?”

This blog post helps answer that question and guides you through our recommended process. This is intended for customers using Office 365 or Exchange Online Protection with on-premises Exchange servers.

First, it is important to understand the difference between an infected and uninfected email. Any email that has an attachment containing a script or malicious executable is considered ‘a virus’ for our purposes. This does not include subscription-based messages with links to malicious sites. Those messages would be considered spam and not viruses, and a different approach is used for spam messages (see this and this).

To deal with an email virus, here are some quick actions that you can perform:

1. Start at the filtering layer. We recommend using EOP, as it is the default option for Office 365 users. However, if you are using a third-party filtering mechanism, you will need to contact your vendor to investigate further. If the email virus has gone through Exchange Online Protection (EOP), or if header information is missing, then proceed to the next step.

2. Create a Transport Rule to block the message. Note that Office 365 Small Business and Small Business Premium customers will not have this feature access as mentioned here. See here also for information about transport and inbox rules limits. If you are an EOP customer, you can perform these steps as mentioned in Microsoft Knowledge Base article 2959596:

• Navigate to Mail Flow in the Exchange Admin Center.
• Click + to create a new rule.
• Click More Options.

Under Apply this rule if, choose any attachment has executable content. At this point you can also choose an action under Do the following. We recommended choosing Block the message…

Make sure no other Transport rules exist that would override this rule. See Manage Transport Rules for more information.

3. Submit the email virus sample immediately to Microsoft Malware Protection Center (MMPC) for further analysis. In order to receive analysis updates, please sign into the MMPC Web site, or enter a valid email address. We recommend you to use your Microsoft account email address.

4. Once you have logged in, select O365 and Exchange Online Protection. Follow the instructions outlined on the MMPC to understand if you need to compress the email virus sample before uploading it to the site. Once you have completed the procedure, make note of the final MMPC ID that will be sent to you from the MMPC Submission Support Team.

If you are dealing with an email virus that has a sender of administrator@domain.com or fax@domain.com with domain.com being same as your Office 365 domain, we also recommend blocking the sending server’s IP address and enabling the SPF hard fail in addition to the steps mentioned above. Also, see Best Practices for Configuring EOP.

If you continue receiving infected messages or attachments, then you should copy the message headers from the email virus, and contact Microsoft Customer Service and Support for further assistance. Be sure to have your MMPC ID handy, as well.

Irol Pinto
Technical Advisor, Microsoft Corporation

EHLO Exchange community,

It seems that PelNet has been well received and I’ve been receiving requests to add much wanted functionality to PelNet. So this article is a quick update on some of the cool new features that administrators can use to help troubleshoot and validate mail flow.

The new features added in this release:

1. Ability to test against multiple recipients – this is useful if you want to test to multiple external domains without having to run the tool again.
2. Optimized remote execution against transport servers for better performance across a large amount of servers.
3. The ability to validate if TLS negotiation is working. This is one of the most useful feature additions.

The above features gave birth to two new parameters validateTLS and CertThumbPrintOverride.

Let’s recap the parameters with the new ones introduced.

• AddressSpace: Which AddressSpace should the script look for in the Send Connectors?
• sendConnector: Specify if you want the scope to be a single Send Connector.
• SourceTransportServers: Accepts comma separated list of transport servers to test from.
• smartHost: The smarthost you want to test against. Accepts comma separated list value. (when validating TLS be sure to use FQDN of remote host – certificate validation will fail if IP is used)
• mailSubmissionTest: Use this switch if you want the script to submit the mail to the mailbox. If you omit the parameter the script will skip the DATA portion of the SMTP verb.
• From: From address (postmaster@contoso.com)
• Rcpt: Recipient Addresses –accepts comma separated list of addresses (testmailbox@fabrikam.com,testmailbox@wingtiptoys.com)
• LogFolderPath: Log file and report location, default will be current path if not specified.
• Port: Default is 25, but you can specify a custom port if you need to.
• validateTLS: This switch will enable TLS validation – this changes the SMTP verb array being used to include the STARTTLS verb (and some other more complicated stuff).
• CertThumbPrintOverride: This allows the operator to override the logic used to determine the SMTP certificate assigned to the transport servers.

  This can also be used to test TLS to a specific host before assigning the certificate to the SMTP service in Exchange, i.e. pre-validations prior to production change. The certificate needs to be in the local machine certificate store.

  It’s important to note that the code logic uses best effort to determine the SMTP certificate assigned. Using the CertThumbPrintOverride parameter allows you to override this easily.

Script Execution Examples

Show the full help with examples

Get-help .\pelnet.ps1 -full

To test mail flow and validate TLS to a smarthost against all the transport servers on port 25 to multiple recipients. (Will not submit the message for delivery)

.\PelNet.ps1 -From postmaster@adatum.com -Rcpt user1@contoso.com,user2@fabrikam.com,user3@wingtiptoys.com -smarthost webmail.contoso.com -validateTLS

To test mail flow and validate TLS to a smarthost against all the transport servers on port 25 to multiple recipients. (Submits the message for delivery and override the certificate to use)

.\PelNet.ps1 -From postmaster@adatum.com -Rcpt user1@contoso.com,user2@fabrikam.com,user3@wingtiptoys.com -smarthost webmail.contoso.com –validateTLS –mailsubmissiontest –certThumbprintOverride 1A13124HJG1234K12JHG312J123D

To test mail flow and validate TLS to EOP from your hybrid servers on port 25 to multiple recipients (also submit the mail to EOP).

.\PelNet.ps1 -From postmaster@adatum.com -Rcpt user1@contoso.com,user2@fabrikam.com,user3@wingtiptoys.com -smarthost adatum-mail-onmicrosoft-com.mail.protection.outlook.com –validateTLS –sourceTransportServers E15HYBRID01,E15HYBRID02 -mailsubmissiontest

The TLS validation logic will authenticate against the remote server with the certificate assigned to SMTP or the certificate that matches the thumbprint used in the override parameter. If the remote host is configured with Opportunistic TLS and the handshake fails the session will fall back to unencrypted SMTP.

The console output won’t show the verb’s being sent as the code is invoking on multiple servers concurrently, but the final output table and output file will be exactly as previously described in PelNet 1.0.

From the below output:

STARTTLS verb being sent with server responding with 2.0.0 SMTP Server Ready and subsequent SSL Stream is established by authenticating against target host using certificate that matches thumbprint provided (or dynamically found using best effort).

Verbs being sent over SSL stream and successful recipient lookup.

The following is an example of a certificate validation issue on one server:

Some of the most common certificate validation errors are:

• Certificate revocation list not found.
• The remote hostname does not match the name on the certificate.
• Certificate is expired.
• The root certificate is not installed on the sending server, i.e. the server does not trust the remote certificate it received.

The above validation error was quickly fixed by installing the root certificate from the Contoso environment on the EX14-02 server.

Until next time,

Michael Hall
Service Engineer
Office 365

The Exchange team is announcing today the availability of Update Rollup 14 for Exchange Server 2007 Service Pack 3. This latest rollup supports recent DST updates. The rollup contains all previously released security bulletins and fixes and updates for Exchange Server 2007 Service Pack 3 as well. This is not a security release, but customers are encouraged to deploy these updates to their environment once proper validation has been completed. More information on this rollup is available in KB2936861.

Note: The KB article may not be fully available at the time this post was published.

The Exchange Team

The Exchange team is announcing today the availability of Update Rollup 7 for Exchange Server 2010 Service Pack 3. Update Rollup 7 is the latest rollup of customer fixes available for Exchange Server 2010 Service Pack 3. The release contains fixes for customer reported issues and previously released security bulletins. Update Rollup 7 is not considered a security release as it contains no new previously unreleased security bulletins. A complete list of issues resolved in Exchange Server 2010 Service Pack 3 Update Rollup 7 may be found in KB2961522. Customers running any Service Pack 3 Update Rollup for Exchange Server 2010 can move to Update Rollup 7 directly.

The release is now available on the Microsoft Download Center. Update Rollup7 will be available on Microsoft Update in September.

Note: The KB article may not be fully available at the time this post was published.

The Exchange Team

Note:: We have learned that customers using Exchange Server 2013 and Exchange Server 2007 co-existence can experience an issue causing Exchange Server 2013 CU6 databases to failover. Please refer to KB2997209 for the specific scenario impacted.

The Exchange team is announcing today the availability of our most recent quarterly servicing update to Exchange Server 2013. Exchange Server 2013 Cumulative Update 6 and updated UM Language Packs are now available on the Microsoft Download Center. CU6 represents the continuation of our Exchange 2013 servicing and builds upon Exchange 2013 CU5. The release includes fixes for customer reported issues, minor product enhancements and previously released security bulletins. A complete list of customer reported issues resolved in Exchange 2013 CU6 can be found in KB 2961810. Customers running any previous release of Exchange 2013 can move directly to CU6 today. Customers deploying Exchange 2013 for the first time may skip previous releases and start their deployment with CU6 as well.

We would like to call your attention to a couple of items in particular about the CU6 release:

• As discussed at MEC 2014 and other forums, CU6 includes significant improvements in Public Folder scalability. More details about this in Public Folder Updates in Exchange 2013 CU6: Improving Scale and More.
• CU6 includes a fix for the HCW issue discussed in KB 2988229. A reminder for those customers who installed the Interim Update which resolved this issue: it's NOT necessary to uninstall the Interim Update prior to installing CU6.

For the latest information and product announcements about Exchange 2013, please read What's New in Exchange 2013, Release Notes and Exchange 2013 documentation on TechNet.

CU6 includes Exchange-related updates to Active Directory schema and configuration. For information on extending schema and configuring Active Directory, please review Prepare Active Directory and domains in Exchange 2013 documentation.

Reminder:  Customers in hybrid deployments where Exchange is deployed on-premises and in the cloud, or who are using Exchange Online Archiving (EOA) with their on-premises Exchange deployment are required to maintain currency on Cumulative Update releases.

Note: Documentation may not be fully available at the time this post was published.

The Exchange Team

Note: Some of the documentation referenced may not be fully available at the time of publishing of this post.

Exchange Server 2013 Cumulative Update 6 (CU6) was released today and provides several important updates for modern Public Folders. This blog post introduces you to the updates delivered in CU6 and discusses our on-going investments in public folders.

10x Increase in Public Folder Limits

CU6 delivers the first round of investments we have made to scale up the limits for Public Folders in Exchange Server 2013. CU6 raises the folder count limit to 100,000 folders. This is a 10x increase over the prior limit as defined in the Exchange Server 2013 limits for public folders. This enables you to migrate and deploy larger Public Folder hierarchies on premises with Exchange Server 2013. Customers can immediately take advantage of this new scale by deploying CU6.

In addition to the increased folder scale capabilities CU6 delivers improvements for concurrent access of Public Folders by reducing lock contention in store for hierarchy sync and content mailbox access.

As you scale the number of public folders in Exchange you might need to keep track of the number of folders that have been created. Exchange PowerShell provides an easy way to see the current public folder count in Exchange Server 2013. Using the Get-PublicFolder and Measure-Object cmdlets you can readily get a current count of your public folders created in Exchange 2013.

Get-PublicFolder –Recurse –ResultSize Unlimited | Measure-Object
Count    : 40051
Average  :
Sum      :
Maximum  :
Minimum  :
Property :

Mail-Enabled Public Folder Permission Changes

This is an important change for customers already using Public Folders in Exchange Server 2013. Prior to CU6 unauthorized senders were able to send messages to mail-enabled Public Folders which means external users could send email to mail-enabled Public Folders regardless of permissions. With CU6, administrators must grant Anonymous users Create Items permissions in the mail-enabled Public Folder to allow external users the ability to send email to the mail-enabled Public Folder. Refer to the CU6 release notes Public Folders section for guidance on updating your configuration.

Automatic Public Folder mailbox readiness management after migration

An additional change delivered in CU6 for Public Folders helps improve the administrator experience post migration. All PF mailboxes serve hierarchy by default in Exchange Server 2013. We have introduced new logic to check if the hierarchy is fully synced to a mailbox after migration. If it is, then the mailbox is automatically made available to serve public folder hierarchy connection else we wait for full sync to complete. This eases work for admin to manually manage readiness of a mailbox after migration completes. Admins can still turn hierarchy serving off and on (default = on) as they please. Refer to the Public Folder article for specific attributes to control hierarchy sync.

Public Folder Deployment Guidance

The increased scale capabilities for public folders in CU6 enables more advanced configurations to begin migrating to modern Public Folders. To meet the needs of these large scale migrations the team has created deployment guidance to assist customers migrating larger scale public folder hierarchies. This deployment guidance will become available in the next few weeks. We will update this post with a link once it is available.

What’s Next

This is the first round of public folder scale improvements we are working to deliver as we shared in the prior Exchange team blog post. Scale improvements are targeted to increase again in a future update. Scale improvements will remain our top priority for Public Folder updates. Other updates such as OWA support for calendar and contacts and Public Folder reporting tools are expected to follow after delivering further scale updates.

Brian Shiers
Technical Product Manager

Frequently asked questions

Q: Is there a limit for the number of sub-folders within a single public folder?
A: The recommended maximum number of sub-folders is 1000. This is the level that has been validated and is the same limit used in Exchange Online.

Q: Is there a limit for the folder depth of the hierarchy?
A: The recommended maximum depth of the hierarchy is 300 folder levels. This is the level that has been validated and is the same limit used in Exchange Online.

Q: Does the increase in folder count change the number of public folder mailboxes or quota for public folders?
A: The number of public folder mailboxes and the total public folder quota remain unchanged. These and other limits are documented in Public Folder Limits article.

Yesterday, I, along with the rest of the Microsoft Exchange community, was shocked and saddened to learn of the untimely death of a colleague and friend, Andrew Ehrensing.

For those that never met Andrew, he was truly one of a kind. With his smile, laughter, and genuine good nature and charm, you were disarmed and put at ease. Everyone that met him became friends with Andrew instantly, and boy could he make you laugh with his stories.

Andrew spent his Microsoft career within Microsoft Consulting Services, working on many of the most challenging Exchange and Lync on-premises projects we could throw at him. Recently, he worked with several customers on the transition to Office 365. He worked tirelessly to meet the expectations of our customers and always over-delivered.

Andrew never shied away from passing on his wisdom, either. He taught during the Certified Master program, delivered many solid presentations at conferences like TechEd and MEC, and always made himself available to help out a friend or colleague with a challenging issue.

He approached everything with a passionate zeal, whether it be with work or in his personal life. He always strived for honesty and for doing right by his family and customers. If he didn’t agree with an approach the company or his friends were taking, he would not only tell us, but provide us the necessary evidence to see the error in our ways and wholeheartedly guide us toward the right path.

This is a terrible loss and our hearts, thoughts, and prayers go out to his wife, children, extended family, and friends.

Andrew will be forever remembered in the annals of Exchange history and in our hearts. If your life happened to be touched by Andrew, please share your anecdotes.  We will collect them and share them with the family.

Rest in peace, Andrew. We miss you.

Microsoft periodically refreshes certificates in Office 365 as part of our effort to maintain a highly available and secure environment. On September 23, 2014, we are making a certificate change on our Microsoft Federation Gateway that could affect some customers as detailed in knowledge base article 2928514. The good news is, you can easily avoid any disruption.

Who is affected?

This certificate change can affect any customer that is using the Microsoft Federation Gateway. If you are in a hybrid configuration or if you are sharing free/busy information between two different on-premises organizations using the Microsoft Federation Gateway as a trust broker, you need to take action.

When will the change occur?

The change is scheduled to occur on September 23, 2014. You must take action before then to avoid any disruption.

What type of issues will you face if no action is taken?

If you don't take action, you won't be able to use services that rely on the Microsoft Federation Gateway. For example:

• A cloud user won't be able to see free/busy information for an on-premises user and vice versa.
• MailTips will not work in a Hybrid configuration.
• Cross-premises free/busy will stop working between organizations that have organization relationships in place.

What action should you take?

If you’re using Exchange Server 2013 SP1 or later no action is required. This is a common task in Exchange 2013 SP1, it happens automatically. Installing the latest version of Exchange Server 2013 will make this an automated task for you.

If you are not running Exchange 2013 SP1 or later, you can create a scheduled task to keep your Federation Trust up-to-date. You can use the following command on your Exchange Server to create a scheduled task to run the update process periodically. This is how we recommend you keep your Federation Trust constantly updated. This will prevent you from being negatively affected by future metadata changes.

Schtasks /create /sc Daily /tn FedRefresh /tr "C:\Windows\System32\WindowsPowerShell\v1.0\powershell.exe -version 2.0 -command Add-PSSnapIn Microsoft.Exchange.Management.PowerShell.E2010;$fedTrust = Get-FederationTrust;Set-FederationTrust -Identity $fedTrust.Name -RefreshMetadata" /ru System

If you prefer to not use a scheduled task, you can manually run the command at any time to refresh the metadata. If you choose a manual option, it is still best practice to update Federation information at least monthly.

Get-Federationtrust | Set-FederationTrust –RefreshMetadata

Jim Lucey

Occasionally I am asked the following question – how can I protect the messaging environment from a rogue administrator? There are essentially two concerns being asked in this question:

1. How do I protect the data from being deleted by a rogue administrator?
2. How do I protect the data from being accessed and/or altered by a rogue administrator?

Sometimes this discussion leads to a discussion about only the chosen backup architecture. The reality is that whether you implement Exchange Native Data Protection or a third-party backup solution, a backup, by itself, does not protect you from rogue administrators; it only mitigates the damage they potentially cause. Any administrator that has the privileged access to the messaging data (whether it be live data and/or backup data), can compromise the system. Therefore, some operational changes must be implemented within the organization in order to reduce the attack surface of an administrator who has gone rogue.

Important: This article is not intended to be a comprehensive set of instructions on how to restrict administrators. Instead, this article will highlight the principles and techniques that can be used.

Immutable Laws of Security

The Microsoft Security Response Center published the Ten Immutable Laws of Security. They are:

• Law #1: If a bad guy can persuade you to run his program on your computer, it's not solely your computer anymore.
• Law #2: If a bad guy can alter the operating system on your computer, it's not your computer anymore.
• Law #3: If a bad guy has unrestricted physical access to your computer, it's not your computer anymore.
• Law #4: If you allow a bad guy to run active content in your website, it's not your website any more.
• Law #5: Weak passwords trump strong security.
• Law #6: A computer is only as secure as the administrator is trustworthy.
• Law #7: Encrypted data is only as secure as its decryption key.
• Law #8: An out-of-date antimalware scanner is only marginally better than no scanner at all.
• Law #9: Absolute anonymity isn't practically achievable, online or offline.
• Law #10: Technology is not a panacea.

These guiding principles are a cornerstone in how we secure Office 365 and are equally applicable to on-premises deployments.

Active Directory

Exchange relies on Active Directory, storing configuration data within the configuration partition and storing recipient data within the domain partitions. The forest administrators, who are members of either the Enterprise Admins group or the root Domain Admins group, control all aspects of the directory and control data stored in the directory (though they sometimes delegate specific rights to others so others can perform very specific actions that are usually narrowly scoped within a forest). Likewise, domain administrators in each domain partition own their respective recipient data. Companies normally restrict forest-wide and domain-wide actions to be exclusively performed by forest administrators because of the risk that configuration changes can have broad adverse impact.

Many organizations today operate an Active Directory model with a least-privilege administrative model. If you are not operating in this fashion, this should be a top priority for your organization. Remember, any member of the Enterprise Admins or Domain Admins can alter messaging configuration settings on recipients and/or the Exchange configuration, without utilizing the Exchange Management Tools. For more information, see Best Practices for Securing Active Directory.

In addition to operating Active Directory in a least-privilege manner, it is important to implement background checks and other processes to determine the trustworthiness of your administrators, otherwise Law #6 cannot be mitigated. In addition, you may want to consider investing in an identity management solution, like Forefront Identity Manager, to manage and audit administrative permission requests and approvals.

Permissions

Exchange administrators have a wide variety of tools that they potentially can utilize to manage a messaging environment. The preferred method is via Remote PowerShellas Exchange can then authorize access and audit any actions performed. However, if an Exchange administrator is granted additional permissions in Active Directory (e.g., the ability to modify any attribute on a user), then the administrator can utilize other tools (e.g., LDP, ADSIEdit, local PowerShell, scripts, etc.) to modify recipient and/or configuration data, bypassing the authorization and auditing checks built into the system.

To effectively mitigate any possibility of modifications that could occur outside of the Remote PowerShell framework, it is imperative to follow Best Practices for Securing Active Directory, as previously mentioned. Often times, Exchange administrators are not evaluated with the same rigor as Active Directory administrators; therefore, Exchange administrators must not be granted permissions in Active Directory that allow them to circumvent Remote PowerShell (e.g., recipient modification).

If Active Directory privileges are accurately and narrowly scoped, a rogue administrator will have difficulty making unauthorized changes no matter which tool he/she tries to use.

PowerShell

Exchange leverages Remote PowerShell which is a feature within PowerShell that lets admins run commands on remote systems. Remote PowerShell enables the functionality utilized by Exchange to audit cmdlet execution and enforce authorization through RBAC.

One way to ensure that administrators can only manage the messaging environment through Remote PowerShell is to not allow the installation of the Exchange Management Tools on administrator workstations. Administrators are then forced to use the Exchange Admin Center or to connect to Exchange using Remote Shell.

In addition, PowerShell 3.0 and higher provides robust audit capabilities when module logging is enabled. PowerShell Module logging captures pipeline execution events for specified modules, and records this data in Windows PowerShellEvent log. Among the events logged, Event ID 800 (Pipeline Execution Details) provides command line commands and a script name if one is used. If a script was used, the ScriptName value will be populated with the file name of the script that was executed and an event will be logged for each line in the script, with each event including the command from that line. The data recorded in the event log can be collected, analyzed, and provide auditing data by which an organization can determine what PowerShell operations are occurring in the environment.

Mitigating Data Destruction

An Exchange administrator has the necessary privilege to access and destroy Exchange data. The best way to protect the messaging data from an administrator is to:

1. Shrink the window of opportunity for administrators to perform malicious activities. The mechanisms that can be implemented include removing local administrative access, deploying Applocker, removing access to destructive cmdlets, and deploying lagged database copies.
2. Ensure all administrative actions are logged and implement alerting and reporting based upon monitoring of logged activities.
3. Implement a least-privilege access control model whereby the elevation process grants access only to perform the intended activities. Even more effective is implementing an access control model whereby most or all administrative activities are done via a ‘control plane’ that is used by administrators to request that actions be performed against Exchange and the control plane can then implement business logic upon the request that will determine if the request will actually be executed. The business logic might include a request to a second person to review and approve the action or it might check an employee work scheduling system to see if the original requestor is on vacation, for example.

Removing Local Administrative Access

The majority of Exchange management tasks are accomplished via Remote PowerShell. As a result, the only time an Exchange administrator needs local administrative access to the Exchange server is to perform system updates (installing driver updates, installing Exchange cumulative updates, operating system updates, etc.). Therefore, local administrative access can be restricted and granted only when needed, for a specific period of time, after which access is revoked.

Without local administrative access, administrators will be unable to obtain certain information about the Exchange server health that they may need to appropriately manage the environment. Therefore, administrators should be granted access to the following:

• File shares (secured via read-only access to the appropriate security groups) can be created to enable access to Exchange diagnostic logs (which by default, are located at %Program Files%\ Exchange Server\V15\Logging and %SystemDrive%\inetpub\logs).
• Read-only access to the event logs can be granted by adding the appropriate security groups to the member server’s local Event Log Readers security group.
• Access to performance counters can be granted by adding the appropriate security groups to the member server’s Performance Monitor Users security group.
• To allow administrators to schedule performance counter logging, enable and collect tracing, the appropriate security groups can be added to the member server’s Performance Log Users security group.

In addition, local administrative access on the administrator’s workstations should also be removed. This ensures that administrators can only run approved applications and cannot bypass any security mechanisms you may put into place to audit and monitor their actions. For more information on running Windows in a least privileged manner, see Applying the Principle of Least Privilege to User Accounts on Windows.

By removing local administrative access, Laws #2 and #3 are mitigated.

AppLocker

AppLocker provides organizations with a strong defense in the prevention of malicious software and unapproved applications from affecting your server environment. AppLocker can be used to limit the software that Exchange operators can use so that only an approved list of applications (e.g., Exchange Management Tools, approved PowerShell scripts, etc.) will run on a server where AppLocker policy enforcement is in effect. For more information, see the TechNet article on AppLocker.

AppLocker allows you to mitigate Law #1.

Removing Access to Destructive Cmdlets

Both Exchange 2010 and Exchange 2013 include Role Based Access Control (RBAC), a mechanism by which administrators are authorized to perform certain administrative tasks. RBAC provides the capability for very granular control of managing the messaging environment – for example, restricting access to a particular cmdlet or to a particular property of a cmdlet.

For more information on RBAC, please see the following articles:

• Understanding Role Based Access Control
• RBAC and the Triangle of Power
• RBAC: Walkthrough of creating a role that can wipe ActiveSync Devices

While Administrator Audit Logging will record the actions taken by administrators, it does not prevent the administrators from performing destructive actions if they are authorized to do so. Using RBAC, organizations can reduce the attack surface area by removing access to destructive cmdlets. A destructive cmdlet is any cmdlet that can access, modify, or delete messaging data.

The below table identifies cmdlets and parameters that may be considered destructive (e.g., Remove-Mailbox). Each cmdlet should be evaluated to determine how it is used in your environment and whether it should be removed from day-to-day usage.

Note: The above table does not include the cmdlet list identified in the section “Removing Access to Data Access Cmdlets” below, but they should also be considered as those cmdlets provide access to messaging data.

For example, if I wanted to ensure that my administrators cannot remove database copies and manipulate the lagged database copy’s configuration, I could remove those cmdlets by using RBAC in the following manner:

1. Create two new role groups: Protected Organization Management and Protected Server Management.

   $ORoleGroup = Get-RoleGroup “Organization Management"
    
   New-RoleGroup "Protected Organization Management" -Roles $ORoleGroup.Roles
    
   $SRoleGroup = Get-RoleGroup “Server Management"
    
   New-RoleGroup "Protected Server Management" -Roles $SRoleGroup.Roles

2. Remove the Database Copies role from the protected role groups.

   Get-ManagementRoleAssignment -RoleAssignee "Protected Organization Management" -Role "Database Copies" -Delegating $false | Remove-ManagementRoleAssignment
    
   Get-ManagementRoleAssignment -RoleAssignee "Protected Server Management" -Role "Database Copies" -Delegating $false | Remove-ManagementRoleAssignment

3. Create a Protected Database Copies role.

   New-ManagementRole –Parent "Database Copies" –Name "Protected Database Copies"

4. Remove the Remove-MailboxDatabaseCopy role entry from the Protected Database Copies role.

   Remove-ManagementRoleEntry "Protected Database Copies\Remove-MailboxDatabaseCopy"

   Additionally, you can also remove access to the ReplayLagTime parameter from the Protected Database Copies role, thereby ensuring administrators cannot disable the lagged database copy.

   Set-ManagementRoleEntry "Protected Database Copies\Set-MailboxDatabaseCopy" –Parameters ReplayLagTime -RemoveParameter

5. Add the Protected Database Copies role to the protected role groups.

   New-ManagementRoleAssignment –SecurityGroup "Protected Organization Management” –Role “Protected Database Copies"
    
   New-ManagementRoleAssignment –SecurityGroup "Protected Server Management” –Role “Protected Database Copies"

6. Add users to the protected role groups.

   $OrgAdmins = Get-RoleGroupMember "Organization Management"
    
   $SrvAdmins = Get-RoleGroupMember "Server Management"
    
   $OrgAdmins | ForEach {Add-RoleGroupMember "Protected Organization Management" –Member $_.Name}
    
   $SrvAdmins | ForEach {Add-RoleGroupMember "Protected Server Management" –Member $_.Name}

7. Remove the desired users from the Organization Management and Server Management role groups.

You can repeat the appropriate steps for each destructive cmdlet that needs to be restricted. Use the following cmdlet to determine which roles contain the desired cmdlets:

Get-ManagementRoleEntry "*\<cmdlet>" | fl name,role

Lagged Database Copies

A lagged database copy is a rolling point-in-time (up to 14 days) copy of the database. Lagged database copies were first introduced in Exchange 2007 and have evolved considerably since then. Lagged database copies are part of The Preferred Architecture. While the primary reason for deploying a lagged database copy is protection against rare, catastrophic logical corruption events, lagged database copies can be used to recover mailboxes and/or items that have been deleted by a rogue administrator. By implementing the access control mechanisms previously discussed, the lagged database copy is protected from destruction by a rogue administrator.

Mitigating Data Access

There are several mechanisms you can implement to mitigate unwarranted data access within your messaging environment. These mechanisms include auditing, using Bitlocker to encrypt the disk drives, and removing access to certain cmdlets that enable administrators to gain access to user data.

Auditing

There are several different forms of auditing that can be implemented in the messaging environment. Within Exchange, you can enable Administrator Audit Logging. Administrator Audit Logging captures all operations that are performed within the Exchange Management Shell (EMS) or Exchange Admin Center (EAC).

By default, all cmdlets, except Get- or Search- cmdlets are logged in the audit logs. You can change the cmdlet logging via Set-AdminAuditLogConfig; for example, since Search-Mailbox includes the ability to delete content, you will want to add that cmdlet to the list. Audit logs are stored in an arbitration mailbox. Reports can be accessed via the Search-AdminAuditLog, New-AdminAuditLogSearch cmdlets, or via EAC.

In addition to auditing the actions taken by Exchange administrators managing the service, you will also want to audit server operation events (e.g., logon events). For more information, see the Windows Server Security Auditing Overview article and the Audit Policy Recommendations article for securing Active Directory.

Bitlocker

Another way to reduce the attack surface area is to use Bitlocker to ensure that operators with physical access to the servers cannot remove disk drives and access the data contained on them.

As mentioned previously, separation of roles is imperative, otherwise Law #7 cannot be mitigated. As Bitlocker recovery information is stored in Active Directory (specifically the msFVE-RecoveryInformation attribute), delegation of this data must not be granted to Exchange administrators.

Removing Access to Data Access Cmdlets

Using RBAC, organizations can reduce the attack surface area by removing access to cmdlets that allow access to mailbox data. The below table identifies cmdlets that grant access to data, or enable administrators to hide their tracks. Each cmdlet should be evaluated to determine how it is used in your environment and whether it should be removed from day-to-day usage.

To remove access to the above cmdlets, follow steps 1-7 from the “Removing Access to Destructive Cmdlets” section.

Requesting Elevation

Over time, administrators will require access to restricted cmdlets to perform a necessary operation (e.g., disabling mailboxes, removing unnecessary transport rules, etc.). There are many ways you could go about this. For example, you could simply build RBAC roles for each individual cmdlet (and/or property) that you restrict; when elevation is required, you add the administrator to the appropriate role group, granting them access, and then remove the administrator when the task is complete. Or you could develop a process and workflow that includes the following:

1. A means by which an administrator may submit a request for elevated access. The request needs to include specifics like the cmdlets being requested, the targeted servers/users/etc., and the length of that time elevated access is required.
2. The request can either be implicitly approved based on the requested action, or it can require human approval.
3. All actions must be logged once elevated access has been granted.
4. Elevated access must be removed once the time period has expired (either based on the request or based on a default time period for access elevations) or the task has been completed.

How is this addressed in Office 365?

Exchange Online implements a variety of mechanisms to prevent rogue administrators from accessing or destroying data.

Perry Clarke and Vivek Sharma recently discussed Lockbox, a mechanism we use to enforce no standing access, in the article From Inside the Cloud: Who has access to your data within Office 365?

In particular, Exchange Online personnel do not have persistent access to the service or servers; instead, administrators (who have undergone background checks) have to request specific access (to servers, cmdlets, etc.) and can only perform the requested tasks during a specified timeframe. Additionally, for requests for elevation to a role with access to customer data, approval must be performed by another human being and the ability to approve this type of request is restricted to a smaller set of more senior personnel.

We also use other mechanisms to protect the service and customer data. For example, Bitlocker is used to encrypt all disks that contain customer data. When the disks are end-of-life they are shredded, thereby ensuring the data cannot be accessed.

Conclusion

While this article covers the basics, there are many other mechanisms you can implement in your environment to reduce the surface attack area within your messaging environment. For more information about security mechanisms protecting Exchange Online that can be used in on-premises deployments, see the MEC session Exchange Online service security investments: you CAN and SHOULD do this at home.

Ross Smith IV
Principal Program Manager
Office 365 Customer Experience

Bulk mail is often mistaken for spam and is starting to become a larger problem for organizations. EOP is not very aggressive out of the box when it comes to bulk mail because this type of mail falls into a grey area. Some organizations will want to receive this type of mail, whereas others will not.

Over the last few months we have greatly increased EOPs ability to detect bulk mail which you can take advantage of starting today. This new system is based on a scale which gives customers the ability to set the aggressiveness of bulk mail detection to meet their specific needs.

X-Microsoft-Antispam is a new header that is stamped on all messages traversing Exchange Online and only started appearing in messages few months ago. This new header currently contains two published values to help better detect bulk and phishing emails.

• BCL – Bulk Complaint Level
• PCL – Phishing Confidence Level

The beauty of this header is that it is stamped on incoming messages BEFORE the EOP transport rules are evaluated. This means EOP transport rules can be written to trigger based on what’s in this header.

One of the goals behind the new X-Microsoft-Antispam header is to allow customers to decide how sensitive they want EOP to be when it comes to bulk mail detection. Currently in the EOP Content Filter there is a bulk mail detection switch that can only be set to either On or Off.

The problem with this switch only being on or off is that bulk mail is a very grey area. What one user considers as bulk another will not. This is why EOP (with no additional configuration added) typically does not block this type of mail. This is also why we are moving beyond the On or Off switch to a multi-value type classification system where customers will be able to set the level that they are comfortable with.

With this new header, you can decide on a scale how sensitive you want the service to be with bulk mail detection. Eventually this will be rolled in to the Advanced Spam Filter options and replace the current bulk On or Off switch, but for now you can write EOP Transport Rules to start taking advantage of this today! You can choose the bulk mail detection level that makes sense for your organization.

At MEC this year there was a great presentation with the title “So how does Microsoft handle my spam?” In this presentation, bulk mail detection is discussed between 22:30 to 28:50 and the speakers provide great insight into this topic. The entire session is great, but I would recommend at least listening to the six minutes where they discuss bulk mail.

What can I do today?

If you are receiving unwanted bulk mail today, the following suggestions can help.

1. Take advantage of the new x-Microsoft-Antispam header by creating an EOP transport rule. The following is an example of a rule that will mark messages as spam if the stamped Bulk Complaint Level is 6 or higher.

For detected messages this rule will set the SCL to 6 which will cause the message to take the spam action you have configured in the content filter. The additional header that this rule adds will make it easy to identify messages that were marked as spam by this rule.
For more information on rules that will increase the bulk sensitivity of EOP see Use transport rules to aggressively filter bulk email messages. This page describes three separate rules, the first of which walks through the creation of the above rule. I would recommend starting only with the first rule that looks at x-Microsoft-Antispam, and if you need even more aggressive filtering, create the subsequent two rules.

2. Educate yourself on the new X-Microsoft-Antispam header. See Anti-spam message headers and Bulk Complaint Level values.

3. Educate your users. If a user recognizes the sender of the bulk message and does not want to receive further mail, they can click the unsubscribe link on the email. If the user does not recognize the sender, they can block the sender or domain in Outlook or OWA by adding the sender to their Blocked Senders list.

4. Submit bulk mail and spam back to Microsoft for analysis. This allows us to continually refine our message filters. See Submitting spam and non-spam messages to Microsoft for analysis.

Note: EOP will always stamp this new header on messages regardless if it already exists or not. This prevents a spammer from manually adding this header themselves and setting a BCL of 0.

Going forward

In the near future it will be easier to take advantage of this new BCL system. We plan to roll this functionality into a slider that will be configurable in the Office 365 portal. Until this happens, creating the transport rule described above will allow you to take advantage of this functionality immediately.

Resources

The following TechNet documentation was updated in July 2014 to include information about the new X-Microsoft-Antispam header.

What’s the difference between junk email and bulk email?
Anti-spam message headers
Bulk Complaint Level values
Use transport rules to aggressively filter bulk email messages

Andrew Stobart