Coming to the next CUs for Exchange 2013 and Exchange 2016 there are some changes to how the Exchange Management Shell (EMS) connects to Exchange. In previous versions, we have seen that customers who were reliant on a load balanced solutions for third party apps and scripts may get routed to a non-Exchange 2016 server. This would lead customers to some broken administrative experiences based on the reliance of Exchange 2016 cmdlets and features. Today we will dive deeper into these changes to help the Exchange Administrator understand how these changes will affect their Exchange Organization.

In Exchange 2010 we used session affinity to provide a long-standing association between a particular client and a particular Client Access Server. With Exchange 2013 and continuing on into Exchange 2016 we have removed the requirement for session affinity. Also with the release of Exchange 2016 we added an ability to proxy between Exchange 2013 and Exchange 2016. This allowed us to have an Exchange 2013 server public facing and proxy traffic back to an Exchange 2016 server. So in order to ensure that when a connection to the Exchange Management Shell (EMS) always goes to an Exchange 2016 server we have introduced mailbox anchoring into Exchange 2013 CU11 and Exchange 2016 CU1.

In mailbox anchoring, CAS locates where the mailbox resides by querying Active Directory for the user's mailbox GUID. As soon as we obtain the GUID, HTTP Proxy refers to Active Manager to locate the database containing the user's mailbox. Once CAS knows where the user's mailbox is located, the protocol request is then proxied to the appropriate mailbox server. If that server goes down and the database is a part of a DAG, the Client Access Proxy will proxy the session to the new active mailbox server for the corresponding database. This process is utilized currently for most client protocols, such as OWA, ECP, and EWS with the exception of Remote PowerShell.

This is great for the client but what about the Exchange Management Shell (EMS)? In Exchange 2013 RTM through CU10 and Exchange 2016 (RTM), we do not use the mailbox anchoring logic to proxy the PowerShell session we just connect to the local server or proxy to another available server in the Exchange Organization. This means if I log into an Exchange 2013 Server I cannot manage any new cmdlets that became available in Exchange 2016. For that I would have to logon directly to an Exchange 2016 Server.

Starting with Exchange Server 2013 CU11 and Exchange Server 2016 CU1, the Exchange Management Shell (EMS) session will also be using mailbox anchoring. If the aforementioned environment has all servers upgraded to Exchange 2013 CU11 and Exchange 2016 CU1 the behavior of Exchange Management Shell changes. Now when a user logs on to the Exchange Server and open up EMS, the session will be proxied to the server where the user’s mailbox is located. If the user does not have a mailbox, we will utilize the arbitration mailboxes for the mailbox anchoring logic. Wherever the arbitration mailbox is mounted is where the EMS session will be proxied.

It is important to understand when you upgrade your existing Exchange Server 2010/Exchange Server 2013 organization to Exchange Server 2013/Exchange 2016, you must move the arbitration mailboxes to a database on highest version of Exchange Mailbox server. Same applies to administrators that have mailbox associated. You should move the mailbox after installing and verifying Exchange was working properly. If you don’t move the arbitration mailbox to higher version of Exchange, you may run into issues like not getting version appropriate tasks or tasks not being saved to the administrator audit log when you run the Search-AdminAuditLog cmdlet. We primarily use the Arbitration mailbox “SystemMailbox{bb558c35-97f1-4cb9-8ff7-d53741dc928c}” however if that mailbox is unavailable we can use the other arbitration mailboxes to make our connection.

So let’s cover how to handle a couple of scenarios that you may run into.

Installing Exchange 2016 into an Exchange 2013 Organization

After you complete the installation of your first Exchange 2016 server and verify that everything is working properly you need to move all arbitration mailboxes to Exchange 2016. To do so you can use the Exchange admin center or Exchange Management Shell and move all of the Microsoft Exchange Migration, Federation, and System Mailboxes.

To move all the arbitration mailboxes via Exchange Management Shell we would run:

[PS] C:\PowerShell> Get-Mailbox -Arbitration | New-MoveRequest -TargetDatabase <Exchange_2016_DataBase>

To move all the arbitration mailboxes via Exchange Admin Center we would run:

1. In the EAC, go to Recipients > Migration.

2. Click New Add Icon, and then click Move to a different database.

3. On the New local mailbox move page, click Select the users that you want to move, and then click Add Icon.

4. On the Select Mailbox page, search for Microsoft Exchange add all the mailboxes that are in the list below:

5. Click OK, and then click Next.

6. On the Move configuration page, type the name of the migration batch, and then click Browse next to the Target database box.

7. On the Select Mailbox Database page, add the mailbox database indicating where you want to move the system mailbox. Verify that the version of the mailbox database that you select is Version 15.1, which indicates that the database is located on an Exchange 2016 server.

8. Click OK, and then click Next.

9. On the Start the batch page, select the options to automatically start and complete the migration request, and then click New.

How do you know which server you are connected to?

The Exchange Management Shell console will always display the name of CAS server that received the initial connection. However, the connection may be proxied to a different mailbox server in the background. You can use following trick to identify the mailbox server to which the connection is being proxied to.

Open up Exchange Management Shell, and run a simple query to determine which Exchange Server you are connected to:

As you can see from the above EMS Session even though I was on the host E1501 when I ran the cmdlet Get-ExchangeCertificate it returned the certificate for server E1503. If you want to dig deeper you can go into the logging directory for the HTTPProxy PowerShell and see what server, you proxied to. (Before you can search in this path you will need to open the path in File Explorer to take ownership)

As you see above we proxied the connection to e1503.contoso.com. Since your PowerShell connection is actually running on the server e1503, therefore all cmdlets run will be against that server. If the cmdlet you are using supports the -server switch, this can be utilized to ensure which server the cmdlet is run against. For example, if I run Get-PowerShellVirtualDirectory I can specify which server I want to communicate with.

Fail Overs

If you have an admin mailbox and the database that hosts the mailbox or DAG that the mailbox is in is down, we will then proxy to the database that has the arbitration mailbox (which is why we recommend moving this mailbox as noted before). This way cmdlets can always be run against the highest version Exchange server in the organization.

AD Site Proxying

In a multisite or geographically dispersed datacenters we will proxy the EMS session back to where the admin mailbox is mounted. In most scenarios this is fine, however there may be some tasks that would require us to have an administrator in the local site. In order to accomplish this, you will need to have either the arbitration mailboxes in the site and logon to the server with an administrator account that has no mailbox or you can create an admin account with a mailbox in the corresponding site.

Critical Failures

If you open EMS and are not able to connect to any Exchange Servers due to loss of communication, then you will need to add the PSSnapin for Exchange to your local PowerShell Session.

Rob Whaley
Beta Engineer for Exchange and Office 365

The Exchange team is announcing the availability of our latest quarterly update for Exchange Server 2013 as well as updates for Exchange Server 2010 Service Pack 3 and Exchange Server 2007 Service Pack 3.

Cumulative Update 11 for Exchange Server 2013 and UM Language Packs are now available on the Microsoft Download Center. Cumulative Update 11 contains the latest set of fixes and builds upon Exchange Server 2013 Cumulative Update 10. The release includes fixes for customer reported issues, minor product enhancements and previously released security bulletins. A complete list of customer reported issues resolved can be found in Knowledge Base Article KB3099522. Customers running any previous release of Exchange Server 2013 can move directly to Cumulative Update 11. Customers deploying Exchange Server 2013 for the first time may skip previous releases and start their deployment with Cumulative Update 11 directly.

Cumulative Update 11 does not include updates to Active Directory Schema, but may add additional RBAC definitions to your existing configuration. PrepareAD should be executed prior to upgrading any servers to CU11. PrepareAD will run automatically during the first server upgrade if Setup detects this is required and the logged on user has sufficient permission.

We are also releasing Update Rollup 12 for Exchange Server 2010 Service Pack 3 (KB3096066). The Exchange Server 2010 update includes a DST update as well as a collection of important fixes. Update Rollup 18 for Exchange Server 2007 Service Pack 3 (KB3078672) is also being released. Update Rollup 18 is primarily a DST fix with an additional opportunistic fix.

We would like to call attention to two particularly important changes included in these updates:

• Cumulative Update 11 and Update Rollup 12 each contain a time zone calculation fix to resolve the issue reported in KB3048372 – Exchange Calendar Items are shifted incorrectly when some Windows DST updates are applied. Customers in impacted time zones are encouraged to deploy these updates to address the issue identified after the release of the OS time zone changes in KB3039024.
• Cumulative Update 11 also includes a change in behavior when deploying/administering Exchange Server 2013 in a co-existent manner with Exchange Server 2010 or 2016. These changes are outlined in the following blog article: Exchange Management Shell and Mailbox Anchoring.

For our customers running Exchange Server 2016, Exchange Server 2016 Cumulative Update 1 will be added to our next set of quarterly updates for Exchange Server.

Microsoft recommends all customers test the deployment of any update in their lab environment to determine the proper installation process for your production environment. For information on extending the schema and configuring Active Directory, please review the appropriate TechNet documentation.

Also, to prevent installation issues you should ensure that the Windows PowerShell Script Execution Policy is set to “Unrestricted” on the server being upgraded or installed. To verify the policy settings, run the Get-ExecutionPolicy cmdlet from PowerShell on the machine being upgraded. If the policies are NOT set to Unrestricted you should use the resolution steps in KB981474 to adjust the settings.

Reminder: Customers in hybrid deployments where Exchange is deployed on-premises and in the cloud, or who are using Exchange Online Archiving (EOA) with their on-premises Exchange deployment are required to deploy the most current (e.g., CU11) or the prior (e.g., CU10) Cumulative Update release.

For the latest information and product announcements please read What’s New in Exchange Server 2013, Release Notes and product documentation available on TechNet.

Note: Documentation may not be fully available at the time this post was published.

The Exchange Team

We are happy to reveal that we’re increasing the number of public folder mailboxes in Exchange from 100 to 1,000.  More Public Folder mailboxes means more storage space capacity for public folders.  This increase will facilitate the migration of very large Public Folders from on-premises Exchange Server to Exchange Online.  This increase has already begun rolling out to Exchange Online tenants, and should be completely deployed by mid-February.

For customers who maintain their own on-premises Exchange Server 2016 infrastructure, the increase is expected to become available via Exchange Server 2016 CU2, in the second quarter of CY2016.  On-premises customers will then have more flexibility regarding placement of data in remote offices where necessary to give primary users a lower latency experience.  It is possible to do this with the existing 100 Public Folder mailbox limit; however, some customers needed to make trade-offs between how many Public Folder mailboxes they could place in regional Exchange locations, and how many they needed for data storage in the main location.  Growing the Public Folder mailbox limit to 1,000 will help those larger customers offer the lowest latency experience to their users, regardless of where they’re located.

Exchange Team

We’re happy to announce the latest addition to the Exchange Server Deployment Assistant!

You can now use the Deployment Assistant to help you configure an Exchange 2016-based hybrid deployment in organizations where Exchange 2010 is already installed. Configuring an Exchange 2016-based hybrid deployment gives you access to features like cross-premises e-discovery, enhanced secure mail, multiple Active Directory forest support, OAuth federation support, and more!

In addition to this new scenario, the following scenarios are already available in the Deployment Assistant:

• Deploy Exchange 2016 into an organization with no existing Exchange deployments
• Upgrade to Exchange 2016 from Exchange 2010

And, we’re working on adding more scenarios to the Deployment Assistant as fast as we can! We’ll post announcements here when they’re available.

In case you're not familiar with it, the Exchange Server Deployment Assistant is a web-based tool that helps you deploy Exchange in your on-premises organization, configure a hybrid deployment between your on-premises organization and Office 365, or migrate to Office 365. Supported on most major browsers, the tool asks you a small set of simple questions and then, based on your answers, creates a customized checklist with instructions to deploy or configure Exchange. Instead of trying to find what you need in the Exchange library, the Deployment Assistant gives you exactly the right information you need to complete your task.

Do you have a deployment success story about the Deployment Assistant? Do you have suggestions on how to improve the tool? We would love your feedback and comments! Feel free to leave a comment here, or send an email to edafdbk@microsoft.com directly or via the 'Feedback' link located in the header of every page of the Deployment Assistant.

Happy deploying!

The Deployment Assistant Team

We wanted to post a quick note to call out that since yesterday, the .NET Framework 4.6.1 has been made a recommended update on WU (Windows Update).

As we have already stated in the Exchange Supportability Matrix, at this time, this version of .NET framework is not supported by Exchange. In fact, we know of some issues if it is installed.

We are working with the .NET team to ensure that Exchange customers have a smooth transition to .NET Framework 4.6.1, but in the meantime, delay this particular .NET update on your Exchange servers (information on how this can be accomplished can be found in the KB article 3133990, How to temporarily block the installation of the .NET Framework 4.6.1).

Update 2/12/2016:

We had several questions on how our customers should go about rolling back to .NET Framework 4.5.2 if 4.6.1 was already automatically installed. Here are the steps:

1. If the server has already automatically updated to 4.6.1 and has not rebooted yet, do so now to allow the installation to complete
2. Stop all running services related to Exchange.  You can run the following cmdlet from Exchange Management Shell to accomplish this:  (Test-ServiceHealth).ServicesRunning | %{Stop-Service $_ -Force}
3. Go to add/remove programs, select view installed updates, and find the entry for KB3102467.  Uninstall the update.  Reboot when prompted.
4. Check the version of the .NET Framework and verify that it is showing 4.5.2.  If it shows a version prior to 4.5.2 go to windows update, check for updates, and install .NET 4.5.2 via the KB2934520 update.  Do NOT select 4.6.1/KB3102467.  Reboot when prompted.  If it shows 4.5.2 proceed to step 5.
5. Stop services using the command from step 2.  Run a repair of .NET 4.5.2 by downloading the offline installer, running setup, and choosing the repair option.  Reboot when setup is complete.
6. Apply the February security updates for .NET 4.5.2 by going to Windows update, checking for updates, and installing KB3122654 and KB3127226.  Do NOT select KB3102467.  Reboot after installation.
7. After reboot verify that the .NET Framework version is 4.5.2 and that security updates KB3122654 and KB3127226 are installed.
8. Follow the steps here to block future automatic installations of .NET 4.6.1.

Thanks to Marc, Nasir and Brett for testing the above to make sure it all works as expected!

Nino Bilic

We are happy to announce the fact that the Exmon tool (aka Microsoft Exchange Server User Monitor) for Exchange 2013 and 2016 is now available!

http://aka.ms/exmon2013

Please note that once you press the Download button, you can also download the updated documentation for this version, in PDF format.

Thanks to Jeff Mealiffe and Nasir Ali for all the work they did to make this possible!

Nino Bilic

The Office 365 Hybrid Configuration wizard has been updated to support Exchange 2010. This new wizard comes with the following advantages:

• An updated user experience that simplifies the hybrid configuration process
• The error handling experience allows for simple remediation of issues, meaning you can actually read and understand the error
• Fixes for HCW can happen quickly and are no longer tied to the on-premises product release cycle
• Inefficient code that caused the HCW to take hours to run has been completely reworked and now you should be in and out in minutes
• Many more enhancements explained in our previous blog post

Please stop using the old HCW for Exchange 2010 that was bundled in the Exchange 2010 Exchange Management Console and instead use the Office 365 Hybrid Configuration wizard.

Video Walkthrough of the new experience

The following is a short video that walks you through the new Office 365 Hybrid Configuration wizard experience for Exchange 2010:

How to run the HCW in your Exchange 2010 environment

In order to run the new Office 365 Hybrid Configuration wizard, you need to change one behavior. Instead of going to the on-premises Exchange Management Console in Exchange 2010, open the Exchange Admin Center in Exchange Online. To do this follow these steps:

1. From a Domain joined machine, go to http://portal.office.com and log in with your tenant administrator credential

2. From the portal landing page select the Admin Icon

3. In the left tree menu of the admin page, expand ADMIN then select Exchange to open the Exchange Admin Center

4. On the Left side of the Exchange Admin Center select the Hybrid node, then select the Configure button to download the wizard

Alternatively, you can click on this link: http://aka.ms/HybridWizard to directly download the new wizard instead.

NOTE: We do plan on updating the Exchange Management Console in Exchange 2010 SP3 RU13 with the new HCW, but there is no reason to wait for that. Just click on the link to the HCW when you are ready to run it, there is no need to even open the EMC.

A few common questions answered:

Why did we update the Office 365 Hybrid Configuration wizard for Exchange 2010?

The new Office 365 Hybrid Configuration wizard, which was released a few months back for Exchange 2013 and 2016, has allowed us to really understand the Hybrid Configuration experience. We can see if there was a failure or slow experience, what the issue was, and we collect and act on any feedback that is provided. All of this telemetry allows us on the engineering side to prioritize and address the issues that need to be addressed quickly. Since we have included Exchange 2010 support in this wizard, now all hybrid customers will see these benefits.

To find out more about the benefits of running the new HCW you can review our previous blog were we introduced the Office 365 Hybrid Configuration wizard.

What Update needs to be installed for the new wizard to work against Exchange 2010?

All you need is Exchange 2010 service pack 3, we do not check for the existence of any rollups. However, newer rollups will have plenty of code and security fixes, so (while not required for HCW to complete) I would make sure you try to stay current. To see the list of the latest updates for each version of Exchange go here.

Can I run the new wizard even though I have already run the old HCW in my environment?

Yes, the new wizard will run even if the old wizard completed or partially completed in your environment. If there is no reason for you to update your old configuration you do not need to run the wizard now, but the next time you have an update to make you should use this new experience.

Is this new hybrid wizard different than the Exchange 2013/2016 hybrid wizard that was announce a few months back?

No, this is the same wizard. However, the wizard has been updated to support the unique configurations that are required for Exchange 2010 Hybrid environments. For example, in a 2010 Hybrid environment you need additional Remote Domain configurations for mail flow features. These Exchange 2010 Explicit configurations needed to be added.

Is this new wizard considered BETA or test software?

No, in fact this is the supported way to configure hybrid moving forward.

Conclusion

Do you want to shave hours off of your hybrid deployment, have a more stable environment, and want to simplify the hybrid configuration experience? Stop using the old Hybrid Configuration wizards and use the new Office 365 Hybrid Configuration wizard instead. If you have an issue or any feedback on the wizard, provide it, there is a feedback button on every page in the wizard and we are eager to read and act on it.

Spread the word to anyone that you know who has run or is getting ready to run the Hybrid Configuration wizard.

Office 365 Hybrid Team

If you’re running Exchange 2013 and you’ve configured a hybrid deployment with Office 365, this post contains important information that might impact you. Please evaluate this information and take any necessary action before April 15, 2016.

On April 15 2016, the Office 365 TLS certificate will be renewed. This certificate is used by Office 365 to provide TLS encryption between Office 365 and external SMTP servers. The new certificate, which will help improve the security of mail sent to and from Office 365, will be issued by a new Certificate Authority and it will have a new Issuer and Subject.

This change has the potential to stop hybrid mailflow between Office 365 and your on-premises Exchange servers if one of the following conditions applies to you:

• Your on-premises Exchange servers are running Exchange 2013 Cumulative Update 8 (CU8) or lower.
• You’ve upgraded the Exchange 2013 servers that handle hybrid mailflow to Exchange 2013 CU9 or higher. However, since upgrading to CU9, you HAVE NOT re-run the Hybrid Configuration wizard (either from the Exchange Admin Center or via the direct download link).

If one of the previous conditions applies to your organization, hybrid mailflow between Office 365 and your organization will stop working after April 15, 2016 unless you complete the steps below.

Note: This only affects hybrid mailflow. Regular mailflow and TLS encryption is NOT affected.

How to keep hybrid mail flowing (MUST be completed before 4/15/2016)

Let the new Hybrid Configuration wizard do it for you

You can use the latest Hybrid Configuration wizard (HCW) to configure your Exchange 2013 servers to work with the new TLS certificate. Just follow these steps:

1. If the Exchange 2013 servers handling hybrid mailflow are running Exchange 2013 CU8 or lower, follow the instructions in Updates for Exchange 2013 to install the latest cumulative update on at least one server.
2. After you install the latest cumulative update, download the new HCW application and run the wizard following the instructions here .

Note: For information on which releases of Exchange are supported with Office 365, see Hybrid deployment prerequisites.

Manual update

If you can’t upgrade Exchange 2013 to latest cumulative update right now (although we would like to remind you of our support policy), you can manually configure your servers to work with the new TLS certificate. On each Exchange 2013 server that’s used for hybrid mailflow, open the Exchange Management Shell, and run the following commands:

$rc=Get-ReceiveConnector |where {$_.TlsDomainCapabilities -like "*<I>*"}

Set-ReceiveConnector -Identity $rc.Identity -TlsDomainCapabilities "mail.protection.outlook.com:AcceptCloudServicesMail

Office 365 Hybrid Team

In Exchange 2013 CU11, we introduced a change to the way Remote PowerShell (RPS) functioned.

Prior to CU11, Exchange 2013 routed Remote PowerShell requests by finding a random mailbox that is either higher than the ExchClientVer that is specified in the URL, or if the ExchClientVer is not specified, by using the current CAS version in which the client connected. We refer to this behavior as server version-based routing.

With CU11, we changed Remote PowerShell to route requests to an anchor mailbox. Typically, this anchor mailbox would be the mailbox of the user attempting the connection. In the event the user attempting the connection did not have a mailbox, the request would be routed to the organization arbitration mailbox.

There were several perceived benefits with this approach:

1. This approach solved an issue with coexistence and ensured that RPS is executed based on the version of the mailbox executing the action. The server version-based routing behavior prior to CU11 led to scenarios where a client could receive the following error:

   New-PSSession : [ems.contoso.com] Processing data from remote server ems.contoso.com failed with the
   following error message: [ClientAccessServer=E2K13-1,BackEndServer=e2k13-1.contoso.com,RequestId=76229690-2343-4
   f-9a51-48184587c5cf,TimeStamp=8/14/2015 2:20:36 PM] [FailureCategory=WSMan-InvalidShellID] The request for the Window
   Remote Shell with ShellId DD266254-5C1F-43C0-A4DA-1797C253C0C0 failed because the shell was not found on the server.
   Possible causes are: the specified ShellId is incorrect or the shell no longer exists on the server. Provide the
   correct ShellId or create a new shell and retry the operation. For more information, see the
   about_Remote_Troubleshooting Help topic.
   At C:\Scripts\test.ps1:8 char:12
   + $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri ht ...
   + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   + CategoryInfo : OpenError: (System.Manageme....RemoteRunspace:RemoteRunspace) [New-PSSession], PSRemoti
   gTransportException
   + FullyQualifiedErrorId : CannotConnectTargetSessionDoesNotExist,PSSessionOpenFailed
   Import-PSSession : Cannot validate argument on parameter 'Session'. The argument is null. Provide a valid value for
   the argument, and then try running the command again.
   At C:\Scripts\test.ps1:10 char:18
   + Import-PSSession $Session
   + ~~~~~~~~
   + CategoryInfo : InvalidData: (:) [Import-PSSession], ParameterBindingValidationException
   + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.ImportPSSessionCommand

   This error occurs when a load balanced RPS request shifts from one version of Exchange to another, whether that be Exchange 2013 coexisting with Exchange 2016, or different versions of cumulative updates for the same version of Exchange.

2. This approach utilized the same code path for Remote PowerShell proxying in Office 365.

Unfortunately, the changes in CU11 introduced several issues with Remote PowerShell in the on-premises world that we did not anticipate. After reviewing each issue and performing an in-depth code review, we made the decision to revert the CU11 change and return to utilizing server version-based routing for Remote PowerShell requests beginning in Exchange 2013 CU12. Exchange 2016 (including CU1) will also use server version-based routing.

If you were planning to take advantage of the changes in Remote PowerShell routing in CU11, you may now be wondering what options are available to you beginning in CU12. Fortunately, you have several options:

1. When connecting to Exchange via Remote PowerShell, specify a server FQDN instead of a load balanced namespace.

   $cred = Get-Credential
   $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://servername.contoso.com/powershell -Authentication Kerberos -Credential $cred
   Import-PSSession $Session

   For more information, please see the TechNet article, Connect to Exchange servers using Remote PowerShell.

2. When connecting to Exchange via Remote PowerShell and specifying a load balanced namespace, specify the ExchClientVer value for the server version you wish to connect against.

   $cred = Get-Credential
   $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri “https://lb.contoso.com/powershell?serializationlevel=Full;ExchClientVer=15.0.225.0" -Authentication Basic -Credential $cred
   Import-PSSession $Session

3. Configure the load balancer to use session affinity for Remote PowerShell requests.
4. Remove the older Exchange versions from the load balanced pool.

We will continue to evaluate ways we can improve the routing of Remote PowerShell requests for on-premises deployments.

Ross Smith IV
Principal Program Manager
Office 365 Customer Experience

The high availability capabilities of the lagged database copy are enhanced in the upcoming release of Exchange 2016 Cumulative Update 1.

ReplayLagManager

As you may recall, lagged copies can care for themselves by invoking automatic log replay to play down the log files in certain scenarios:

• When a low disk space threshold (10,000MB) is reached
• When the lagged copy has physical corruption and needs to be page patched
• When there are fewer than three available healthy HA copies for more than 24 hours

Play down based on health copy status requires ReplayLagManager to be enabled. Beginning with Exchange 2016 CU1, ReplayLagManager is enabled by default. You can change this via the following command:

Set-DatabaseAvailabilityGroup <DAGName> -ReplayLagManagerEnabled $false

Deferred Lagged Copy Play Down

When one of the above conditions is triggered, the Replication Service will initiate a play down event for the lagged database copy. However, there are times where this may not be ideal. For example, consider the scenario where there are four database copies on a disk, one passive, one lagged, and two active. Initiating a play down event on the lagged copy has the potential to impact any active copies on that disk – replaying log files generates IO and introduces disk latency as the disk head moves, which impacts users accessing their data on the active copies.

To address this concern, beginning with Cumulative Update 1 for Exchange 2016, the lagged copy’s play down activity is tied to the health of the disk. When a play down event is initiated, the disk’s IO latency is evaluated:

1. If the disk’s read IO latency is above 25ms, the play down event is deferred. In the event that there is a disk capacity concern, the disk latency deferral will be ignored and the lagged copy will play down.
2. If the disk’s read IO latency is below 25ms, the play down event is allowed.

As a result, deferred lagged copy play down reduces the IO burstiness of lagged copy play down events and ensures that local active copies on the lagged copies disk are not affected. IO sizing of a lagged database copy does not change with this feature (nor does it affect the IO sizing of an active copy); you still must ensure there is available IO headroom in the event that the lagged copy becomes active.

Consider the following example:

The y axis is disk latency, measured in milliseconds. The x axis is a 24-hour period.

As you can see from the graph, between the hours of 1am to 9am, the disk IO latency is below 25ms, meaning that lagged copy replay is allowed. At 10am, the latency exceeds 25ms and this continues until about 2pm; during this time period, lagged copy replay is delayed or deferred. At 2pm, the latency drops below 25ms and lagged copy replay resumes. Latency increases again at 3pm and the process repeats itself.

By default, the maximum amount of time that a play down event can be deferred is 24 hours. You can adjust this via the following command:

Set-MailboxDatabaseCopy <database name\server> -ReplayLagMaxDelay:<value in the format of 00:00:00>

If you want to disable deferred play down, you can set the ReplayLagMaxDelay value to 00:00:00.

The following events are recorded in the Microsoft-Exchange-HighAvailability/Monitoring crimson channel when log replay is deferred or resumed:

• Event 750 – Replay Lag Manager requested activating replay lag delay (suspending log replay) for database copy '%1\%2' after a suppression interval of %4. Delay Reason: %6"
• Event 751 – Replay Lag Manager successfully activated replay lag delay (suspended log replay) for database copy '%1\%2'. Delay Reason: %4"
• Event 752 – Replay Lag Manager failed to activate replay lag delay (suspend log replay) for database copy '%1\%2'. Error: %4"
• Event 753 – Replay Lag Manager requested deactivating replay lag (resuming log replay) for database copy '%1\%2' after a suppression interval of %4. Reason: %5"
• Event 754 – Replay Lag Manager successfully deactivated replay lag (resumed log replay) for database copy '%1\%2'. Reason: %4
• Event 755 - Replay Lag Manager failed to deactivate replay lag (resume log replay) for database copy '%1\%2'. Error: %4
• Event 756 - Replay Lag Manager will attempt to deactivate replay lag (resume log replay) for database copy '%1\%2' because it has reached the maximum allowed lag duration. Detailed Reason: %5

The following events are recorded in the Microsoft-Exchange-HighAvailability/Operational crimson channel when log replay is deferred or resumed:

• Event 748 – Log Replay suspend/resume state for database '%1' has changed. (LastSuspendReason=%3, CurrentSuspendReason=%4, CurrentSuspendReasonMessage=%5)
• Event 2050 – Suspend log replay requested for database guid=%1, reason='%2'.
• Event 2051 – Suspend log replay for database guid=%1 succeeded.
• Event 2052 – Suspend log replay for database guid=%1 failed: %2.
• Event 2053 – Resume log replay requested for database guid=%1.
• Event 2054 – Resume log replay for database guid=%1 succeeded.
• Event 2055 – Resume log replay for database guid=%1 failed: %2.

Summary

The changes discussed above continue our work in improving the Preferred Architecture by ensuring that users have the best possible experience on the Exchange platform.

As always, we welcome your feedback.

Ross Smith IV
Principal Program Manager
Office 365 Customer Experience

Did you know that today is 20 years since we declared RTM (Released to Manufacturing) of Exchange 4.0?

Okay, so maybe this did not happen in the galaxy far away, but it sure feels like it! Here is a great overview of Exchange's earlier years (up to Exchange 2007).

Exchange has been evolving for over 20 years now, and is still going strong! From our humble beginnings, adding SMTP protocol through an IMC connector as well as web access to email in Exchange 5.0 back in 1997 (that was some cutting edge stuff back then!), starting to play in service waters 10 years later in 2007 (ever heard of Exchange Labs?) all the way to where we are today with Exchange Server 2016 and Exchange Online– we have gone through many a transformation.

One thing is certain, though – we would not have been here without you, our customers. You keep pushing us to get better and bring you more features. When we do something wrong (yeah, that happens), the incredible Exchange community is always quick to let us know. And for those of you wishing that this virtual birthday celebration could somehow take place in person, you’ll want to sign up to attend Microsoft Ignite in Atlanta.

Thank you for all the support and amazing ride! Anyone up for 20 more?

(Hat tip, Tony.)

The Exchange Team

The Exchange team is happy to announce our spring quarterly updates for Exchange Server are now available on the Microsoft Download Center. Exchange Server 2016 receives its first Cumulative Update, and Exchange Server 2013 Cumulative Update 12 is also released. Exchange Server 2007 and Exchange Server 2010 Update Rollups provide an updated OWA S/MIME control signed with a SHA-2 certificate. More information and highlights of all these releases can be found below.

Updated OWA S/MIME control

All of the packages released today include an update to the OWA S/MIME control. The control itself has not changed, but has now been signed with a SHA-2 compliant certificate. All of the updates released will install the updated control onto the Exchange Server. Users who have installed the control into their browser will need to re-install this onto devices where the previous version was installed. Installing the control is straight forward and can be done quickly using OWA Options, Exchange Control Panel or Exchange Admin Center depending upon the release of Exchange you are using.

New distribution package for Exchange Server 2016 updates

With the introduction of Cumulative Updates for Exchange Server 2016, we are making a change to the update package type for this product version. Previous versions of Exchange used self-extracting packages to deliver service packs and cumulative updates. We have heard requests to release these updates as .ISO’s. With the capability to mount .ISO’s directly in Windows Server 2012 and later, we think it makes sense to ship Cumulative Updates as .ISO’s. At this time, we are not planning to do this for Exchange Server 2013 Cumulative Updates but could be persuaded to do so if enough people ask for it. One down side to this approach is that the package is much larger. However, copying a single .ISO vs. the ever growing number of files and folders over the network is much more efficient and faster. We hope you like this change.

Change to Mailbox Anchoring for Remote PowerShell

We heard your feedback on the changes to load balancing Remote PowerShell introduced into Exchange Server 2013 and 2016. As announced by Ross here, we have reverted this behavior in the Cumulative Updates being released today.

Additional languages for Outlook on the Web

Exchange Server 2016 Cumulative Update 1 adds support for 17 additional languages in Outlook on the Web. These languages will appear automatically in the language selection drop down after a server is updated to Cumulative Update 1.

.Net 4.6.1 Support

We know that many of you have been asking about .Net 4.6.1 and Exchange. Rest assured we are working closely with the .Net Framework team to resolve issues preventing us from supporting .Net 4.6.1 with Exchange Server. While we are not there yet, we hope to be very soon. Support for .Net 4.6.1 is planned for future Cumulative Updates for Exchange Server 2013 and 2016.

Slow installations on Windows Server 2012 R2

For customers who are running Exchange on Windows Server 2012 R2, we want to make certain you are aware of a condition which can substantially increase the amount of time it takes to install Exchange Updates on this OS. Working with the .Net team, we have discovered that systems which have applied Windows Update KB3097966 can take 50% more time to install Exchange. The .Net team is working on a resolution to this and will include a fix in a future product update. In the meantime, customers who have deployed this Windows update can take a one-time action on their server before installing Exchange or a Cumulative Update to bring installation time back to normal. This procedure needs to be done once on every Exchange server running Windows Server 2012 R2. The command to execute is:

“%windir%\Microsoft.NET\Framework64\v4.0.30319\ngen.exe update”

Errors and warnings encountered running this command can be safely ignored provided the final exit status code of 0 is reported in the output.

Support for Standalone Hybrid Configuration Wizard in Exchange Server 2010

Customers using Exchange Server 2010 in Hybrid mode with Office 365 will notice a new link in the EMC to use the Updated Standalone Hybrid Configuration Wizard. We encourage all customers to use this updated version of the Hybrid Configuration Wizard.

Release Details

KB articles which contain greater depth on what each release includes are available as follows:

• Exchange Server 2016 Cumulative Update 1 (KB3134844), Download, UM Lang Packs
• Exchange Server 2013 Cumulative Update 12 (KB3108023), Download, UM Lang Packs
• Exchange Server 2010 Service Pack 3 Update Rollup 13 (KB3141339), Download
• Exchange Server 2007 Service Pack 3 Update Rollup 19 (KB3141352), Download

Note: Documentation may not be fully available at the time this post was published.

Exchange Server 2016 Cumulative Update 1 does include updates to Active Directory Schema. These updates will apply automatically during setup if the permissions and AD requirements are met during installation. If the Exchange Administrator lacks permissions to update Active Directory Schema, a Schema Admin should execute SETUP /PrepareSchema before installing Cumulative Update 1 on your first server. The Exchange Administrator should also execute SETUP /PrepareAD to ensure RBAC roles are updated correctly.

Exchange Server 2013 Cumulative Update 12 does not include updates to Active Directory, but may add additional RBAC definitions to your existing configuration. PrepareAD should be executed prior to upgrading any servers to CU12. PrepareAD will run automatically during the first server upgrade if Setup detects this is required and the logged on user has sufficient permission.

Additional Information

Microsoft recommends all customers test the deployment of any update in their lab environment to determine the proper installation process for your production environment. For information on extending the schema and configuring Active Directory, please review the appropriate TechNet documentation.

Also, to prevent installation issues you should ensure that the Windows PowerShell Script Execution Policy is set to “Unrestricted” on the server being upgraded or installed. To verify the policy settings, run the Get-ExecutionPolicy cmdlet from PowerShell on the machine being upgraded. If the policies are NOT set to Unrestricted you should use the resolution steps in KB981474 to adjust the settings.

Reminder: Customers in hybrid deployments where Exchange is deployed on-premises and in the cloud, or who are using Exchange Online Archiving (EOA) with their on-premises Exchange deployment are required to deploy the most current (e.g., CU12) or the prior (e.g., CU11) Cumulative Update release.

For the latest information on Exchange Server and product announcements please see What's New in Exchange Server 2016 and Exchange Server 2016 Release Notes. You can also find updated information on Exchange Server 2013 in What’s New in Exchange Server 2013, Release Notes and product documentation available on TechNet.

The Exchange Team

If you’re an Exchange Online or Exchange Online Protection (EOP) subscriber and you have configured connectors, this post contains important information that might impact your organization. To make sure that your mail flow isn’t interrupted, we strongly recommend that you read this post and take any necessary action at your earliest convenience.

The change will impact you if one of the following scenarios apply to your organization:

• Your organization needs to send NDR (non-delivery report) messages to a recipient on the Internet and needs to relay them through Office 365.
• Your organization needs to send messages from your own email server (on-premises environment) from domains that your organization has not entered in Office 365 (see Add Domains in Office 365). For example, your organization Contoso needs to send email as the domain fabrikam.com, which doesn’t belong to your organization.
• There is a forwarding rule configured on your on-premises server, and messages need to relay through Office 365. For example, contoso.com is your organization’s domain, a user in your organization’s on-premises server, kate@contoso.com, has enabled forwarding. All her messages go to kate@tailspintoys.com. If john@fabrikam.com sends a message to kate@contoso.com, the message gets automatically forwarded to kate@tailspintoys.com. From Office 365’s point of view, the message is sent from john@fabrikam.com to kate@tailspintoys.com. Because Kate’s mail is being forwarded, neither the sender domain nor the recipient domain belongs to your organization.

Beginning February 1, 2017, Office 365 will no longer by default support relaying messages for the scenarios described above. If your organization needs those scenarios to continue to work, you need to make sure that the following are all true:

• You have created a connector in Office 365 that instructs the service to use certificate to authenticate emails coming from your organization’s own email server (on-premises environment).
• Your own email server (on-premises environment) is configured to use the certificate to send email to Office 365.
• This certificate is CA signed and its certificate name (CN) or subject alternative name (SAN) contains a domain that you have entered in Office 365.

To do so, use the following instructions.

Create or Edit a certificate-based connector in Office 365

For Office 365 to relay messages to internet that match with the scenarios listed above, you need to follow the below steps.

1. Sign in to Office 365 admin center, and go to Admin> Exchange.

2. Go to mail flow> connectors, and do one of the following:

If there are no connectors, choose ’+’ (Add) to create a connector.

If a connector already exists, select the connector, and choose Edit to modify it.

3. On the Select your mail flow scenario page, choose From: Your organization’s email server and To: Office 365. This creates a connector that indicates that your on-premises server is the sending source for your messages.

4. Enter connector name and other information, and then choose Next.

5. On the New connector or Edit connector page, choose the first option to use a TLS certificate to identify the sender source of your organization’s messages. The domain name in the option should match with the CN name or SAN in the certificate that you’re using. The domain you use needs to be a domain that belongs to your organization and you need to have added the domain to Office 365. For example, contoso.com belongs to your organization, and it’s part of CN name or SAN name in the certificate that your organization uses to communicate with Office 365.

Configure your on-premises environment

Use the following steps to prepare your on-premises servers to relay messages through Office 365:

1. If your organization uses Exchange server for its on-premises server, you need to configure your server to send messages over TLS. To do this, follow Set up your email server to relay mail to the Internet via Office 365, which is part 2.2 of “Set up connectors to route mail between Office 365 and your own email servers.” If you have already used Hybrid Configuration Wizard, then continue to use it, but ensure to use a certificate that matches the criteria outlined in step 5 of the previous section.
2. Install a certificate in your on-premises environment. For details, follow “Step 6: Configure an SSL certificate” of Configure mail flow and client access.

For more details about how to relay messages through Office 365, see the Setting up mail flow where some mailboxes are in Office 365 and some mailboxes are on your organization’s mail servers section of Mail flow best practices for Exchange Online and Office 365.

Carolyn Liu

Today marks the start of the one-year countdown before Exchange Server 2007 reaches the end of extended support. If Exchange Server 2007 is still part of your messaging infrastructure, it’s not too early to start planning an update.

It’s hard to believe that it’s been 9+ years since we released CCR, LCR and SCR. These technologies of course laid the ground work for the Database Availability groups we’ve relied upon since Exchange Server 2010. Exchange Server 2007 also marked the start of the transition to building Exchange Server on .Net Frameworks as well. We have continued that investment and the .Net Frameworks is now the foundation of all critical Exchange processes in Exchange Server 2013, 2016 and Office 365. PowerShell, also new to Exchange Server 2007, is even more prevalent in current versions of Exchange and is the de facto management tool for modern Exchange Servers.

As revolutionary as Exchange Server 2007 was at the time, our latest versions of Exchange Server and Office 365 have even more to offer. Customers running Exchange Server 2007 have the option to upgrade via mailbox move to Exchange Server 2010, 2013 or migrate directly to Office 365. Customers wanting to migrate to our latest version of Exchange Server, Exchange Server 2016, will need to first decrement Exchange Server 2007. Customers wanting to maximize their on-premises server investment should strongly consider migrating to Exchange Server 2016 as Exchange Server 2013 is already three years into its own 10-year lifecycle.

Below are links which you may find helpful to start planning your migration off of Exchange Server 2007 and be on your way to experiencing the latest capabilities of Exchange Server.

• Microsoft Exchange Server Deployment Assistant
• Upgrade from Exchange Server 2007 to Exchange 2013
• Planning and Deployment of Exchange Server 2016
• Office 365 Fast Track Center

The Exchange Team

We are happy to announce the fact that the Exmon tool (aka Microsoft Exchange Server User Monitor) for Exchange 2013 and 2016 is now available!

http://aka.ms/exmon2013

Please note that once you press the Download button, you can also download the updated documentation for this version, in PDF format.

Thanks to Jeff Mealiffe and Nasir Ali for all the work they did to make this possible!

Nino Bilic

The Office 365 Hybrid Configuration wizard has been updated to support Exchange 2010. This new wizard comes with the following advantages:

• An updated user experience that simplifies the hybrid configuration process
• The error handling experience allows for simple remediation of issues, meaning you can actually read and understand the error
• Fixes for HCW can happen quickly and are no longer tied to the on-premises product release cycle
• Inefficient code that caused the HCW to take hours to run has been completely reworked and now you should be in and out in minutes
• Many more enhancements explained in our previous blog post

Please stop using the old HCW for Exchange 2010 that was bundled in the Exchange 2010 Exchange Management Console and instead use the Office 365 Hybrid Configuration wizard.

Video Walkthrough of the new experience

The following is a short video that walks you through the new Office 365 Hybrid Configuration wizard experience for Exchange 2010:

How to run the HCW in your Exchange 2010 environment

In order to run the new Office 365 Hybrid Configuration wizard, you need to change one behavior. Instead of going to the on-premises Exchange Management Console in Exchange 2010, open the Exchange Admin Center in Exchange Online. To do this follow these steps:

1. From a Domain joined machine, go to http://portal.office.com and log in with your tenant administrator credential

2. From the portal landing page select the Admin Icon

3. In the left tree menu of the admin page, expand ADMIN then select Exchange to open the Exchange Admin Center

4. On the Left side of the Exchange Admin Center select the Hybrid node, then select the Configure button to download the wizard

Alternatively, you can click on this link: http://aka.ms/HybridWizard to directly download the new wizard instead.

NOTE: We do plan on updating the Exchange Management Console in Exchange 2010 SP3 RU13 with the new HCW, but there is no reason to wait for that. Just click on the link to the HCW when you are ready to run it, there is no need to even open the EMC.

A few common questions answered:

Why did we update the Office 365 Hybrid Configuration wizard for Exchange 2010?

The new Office 365 Hybrid Configuration wizard, which was released a few months back for Exchange 2013 and 2016, has allowed us to really understand the Hybrid Configuration experience. We can see if there was a failure or slow experience, what the issue was, and we collect and act on any feedback that is provided. All of this telemetry allows us on the engineering side to prioritize and address the issues that need to be addressed quickly. Since we have included Exchange 2010 support in this wizard, now all hybrid customers will see these benefits.

To find out more about the benefits of running the new HCW you can review our previous blog were we introduced the Office 365 Hybrid Configuration wizard.

What Update needs to be installed for the new wizard to work against Exchange 2010?

All you need is Exchange 2010 service pack 3, we do not check for the existence of any rollups. However, newer rollups will have plenty of code and security fixes, so (while not required for HCW to complete) I would make sure you try to stay current. To see the list of the latest updates for each version of Exchange go here.

Can I run the new wizard even though I have already run the old HCW in my environment?

Yes, the new wizard will run even if the old wizard completed or partially completed in your environment. If there is no reason for you to update your old configuration you do not need to run the wizard now, but the next time you have an update to make you should use this new experience.

Is this new hybrid wizard different than the Exchange 2013/2016 hybrid wizard that was announce a few months back?

No, this is the same wizard. However, the wizard has been updated to support the unique configurations that are required for Exchange 2010 Hybrid environments. For example, in a 2010 Hybrid environment you need additional Remote Domain configurations for mail flow features. These Exchange 2010 Explicit configurations needed to be added.

Is this new wizard considered BETA or test software?

No, in fact this is the supported way to configure hybrid moving forward.

Conclusion

Do you want to shave hours off of your hybrid deployment, have a more stable environment, and want to simplify the hybrid configuration experience? Stop using the old Hybrid Configuration wizards and use the new Office 365 Hybrid Configuration wizard instead. If you have an issue or any feedback on the wizard, provide it, there is a feedback button on every page in the wizard and we are eager to read and act on it.

Spread the word to anyone that you know who has run or is getting ready to run the Hybrid Configuration wizard.

Office 365 Hybrid Team

If you’re running Exchange 2013 and you’ve configured a hybrid deployment with Office 365, this post contains important information that might impact you. Please evaluate this information and take any necessary action before April 15, 2016. If your latest run of the Hybrid Configuration Wizard was initiated from Exchange 2010 than you are NOT affected.

Note: This information is now also published in KB3145044.

On April 15 2016, the Office 365 TLS certificate will be renewed. This certificate is used by Office 365 to provide TLS encryption between Office 365 and external SMTP servers. The new certificate, which will help improve the security of mail sent to and from Office 365, will be issued by a new Certificate Authority and it will have a new Issuer and Subject.

This change has the potential to stop hybrid mailflow between Office 365 and your on-premises Exchange servers if one of the following conditions applies to you:

• Your on-premises Exchange servers are running Exchange 2013 Cumulative Update 8 (CU8) or lower.
• You’ve upgraded the Exchange 2013 servers that handle hybrid mailflow to Exchange 2013 CU9 or higher. However, since upgrading to CU9, you HAVE NOT re-run the Hybrid Configuration wizard (either from the Exchange Admin Center or via the direct download link).

If one of the previous conditions applies to your organization, hybrid mailflow between Office 365 and your organization will stop working after April 15, 2016 unless you complete the steps below.

Note: This only affects hybrid mailflow. Regular mailflow and TLS encryption is NOT affected.

How to keep hybrid mail flowing (MUST be completed before 4/15/2016)

Let the new Hybrid Configuration wizard do it for you

You can use the latest Hybrid Configuration wizard (HCW) to configure your Exchange 2013 servers to work with the new TLS certificate. Just follow these steps:

1. If the Exchange 2013 servers handling hybrid mailflow are running Exchange 2013 CU8 or lower, follow the instructions in Updates for Exchange 2013 to install the latest cumulative update on at least one server.
2. After you install the latest cumulative update, download the new HCW application and run the wizard following the instructions here .

Note: For information on which releases of Exchange are supported with Office 365, see Hybrid deployment prerequisites.

Manual update

If you can’t upgrade Exchange 2013 to latest cumulative update right now (although we would like to remind you of our support policy), you can manually configure your servers to work with the new TLS certificate. On each Exchange 2013 server that’s used for hybrid mailflow, open the Exchange Management Shell, and run the following commands:

$rc=Get-ReceiveConnector |where {$_.TlsDomainCapabilities -like "*MSIT Machine Auth CA 2*"}

$rc | foreach {Set-ReceiveConnector -Identity $_.identity -TlsDomainCapabilities "mail.protection.outlook.com:AcceptCloudServicesMail”}

Office 365 Hybrid Team

In Exchange 2013 CU11, we introduced a change to the way Remote PowerShell (RPS) functioned.

Prior to CU11, Exchange 2013 routed Remote PowerShell requests by finding a random mailbox that is either higher than the ExchClientVer that is specified in the URL, or if the ExchClientVer is not specified, by using the current CAS version in which the client connected. We refer to this behavior as server version-based routing.

With CU11, we changed Remote PowerShell to route requests to an anchor mailbox. Typically, this anchor mailbox would be the mailbox of the user attempting the connection. In the event the user attempting the connection did not have a mailbox, the request would be routed to the organization arbitration mailbox.

There were several perceived benefits with this approach:

1. This approach solved an issue with coexistence and ensured that RPS is executed based on the version of the mailbox executing the action. The server version-based routing behavior prior to CU11 led to scenarios where a client could receive the following error:

   New-PSSession : [ems.contoso.com] Processing data from remote server ems.contoso.com failed with the
   following error message: [ClientAccessServer=E2K13-1,BackEndServer=e2k13-1.contoso.com,RequestId=76229690-2343-4
   f-9a51-48184587c5cf,TimeStamp=8/14/2015 2:20:36 PM] [FailureCategory=WSMan-InvalidShellID] The request for the Window
   Remote Shell with ShellId DD266254-5C1F-43C0-A4DA-1797C253C0C0 failed because the shell was not found on the server.
   Possible causes are: the specified ShellId is incorrect or the shell no longer exists on the server. Provide the
   correct ShellId or create a new shell and retry the operation. For more information, see the
   about_Remote_Troubleshooting Help topic.
   At C:\Scripts\test.ps1:8 char:12
   + $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri ht …
   + ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
   + CategoryInfo : OpenError: (System.Manageme….RemoteRunspace:RemoteRunspace) [New-PSSession], PSRemoti
   gTransportException
   + FullyQualifiedErrorId : CannotConnectTargetSessionDoesNotExist,PSSessionOpenFailed
   Import-PSSession : Cannot validate argument on parameter 'Session'. The argument is null. Provide a valid value for
   the argument, and then try running the command again.
   At C:\Scripts\test.ps1:10 char:18
   + Import-PSSession $Session
   + ~~~~~~~~
   + CategoryInfo : InvalidData: (:) [Import-PSSession], ParameterBindingValidationException
   + FullyQualifiedErrorId : ParameterArgumentValidationError,Microsoft.PowerShell.Commands.ImportPSSessionCommand

   This error occurs when a load balanced RPS request shifts from one version of Exchange to another, whether that be Exchange 2013 coexisting with Exchange 2016, or different versions of cumulative updates for the same version of Exchange.

2. This approach utilized the same code path for Remote PowerShell proxying in Office 365.

Unfortunately, the changes in CU11 introduced several issues with Remote PowerShell in the on-premises world that we did not anticipate. After reviewing each issue and performing an in-depth code review, we made the decision to revert the CU11 change and return to utilizing server version-based routing for Remote PowerShell requests beginning in Exchange 2013 CU12. Exchange 2016 (including CU1) will also use server version-based routing.

If you were planning to take advantage of the changes in Remote PowerShell routing in CU11, you may now be wondering what options are available to you beginning in CU12. Fortunately, you have several options:

1. When connecting to Exchange via Remote PowerShell, specify a server FQDN instead of a load balanced namespace.

   $cred = Get-Credential
   $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri https://servername.contoso.com/powershell -Authentication Kerberos -Credential $cred
   Import-PSSession $Session

   For more information, please see the TechNet article, Connect to Exchange servers using Remote PowerShell.

2. When connecting to Exchange via Remote PowerShell and specifying a load balanced namespace, specify the ExchClientVer value for the server version you wish to connect against.

   $cred = Get-Credential
   $Session = New-PSSession -ConfigurationName Microsoft.Exchange -ConnectionUri “https://lb.contoso.com/powershell?serializationlevel=Full;ExchClientVer=15.0.225.0" -Authentication Basic -Credential $cred
   Import-PSSession $Session

3. Configure the load balancer to use session affinity for Remote PowerShell requests.
4. Remove the older Exchange versions from the load balanced pool.

We will continue to evaluate ways we can improve the routing of Remote PowerShell requests for on-premises deployments.

Ross Smith IV
Principal Program Manager
Office 365 Customer Experience

The high availability capabilities of the lagged database copy are enhanced in the upcoming release of Exchange 2016 Cumulative Update 1.

ReplayLagManager

As you may recall, lagged copies can care for themselves by invoking automatic log replay to play down the log files in certain scenarios:

• When a low disk space threshold (10,000MB) is reached
• When the lagged copy has physical corruption and needs to be page patched
• When there are fewer than three available healthy HA copies for more than 24 hours

Play down based on health copy status requires ReplayLagManager to be enabled. Beginning with Exchange 2016 CU1, ReplayLagManager is enabled by default. You can change this via the following command:

Set-DatabaseAvailabilityGroup <DAGName> -ReplayLagManagerEnabled $false

Deferred Lagged Copy Play Down

When one of the above conditions is triggered, the Replication Service will initiate a play down event for the lagged database copy. However, there are times where this may not be ideal. For example, consider the scenario where there are four database copies on a disk, one passive, one lagged, and two active. Initiating a play down event on the lagged copy has the potential to impact any active copies on that disk – replaying log files generates IO and introduces disk latency as the disk head moves, which impacts users accessing their data on the active copies.

To address this concern, beginning with Cumulative Update 1 for Exchange 2016, the lagged copy’s play down activity is tied to the health of the disk. When a play down event is initiated, the disk’s IO latency is evaluated:

1. If the disk’s read IO latency is above 25ms, the play down event is deferred. In the event that there is a disk capacity concern, the disk latency deferral will be ignored and the lagged copy will play down.
2. If the disk’s read IO latency is below 25ms, the play down event is allowed.

As a result, deferred lagged copy play down reduces the IO burstiness of lagged copy play down events and ensures that local active copies on the lagged copies disk are not affected. IO sizing of a lagged database copy does not change with this feature (nor does it affect the IO sizing of an active copy); you still must ensure there is available IO headroom in the event that the lagged copy becomes active.

Consider the following example:

The y axis is disk latency, measured in milliseconds. The x axis is a 24-hour period.

As you can see from the graph, between the hours of 1am to 9am, the disk IO latency is below 25ms, meaning that lagged copy replay is allowed. At 10am, the latency exceeds 25ms and this continues until about 2pm; during this time period, lagged copy replay is delayed or deferred. At 2pm, the latency drops below 25ms and lagged copy replay resumes. Latency increases again at 3pm and the process repeats itself.

By default, the maximum amount of time that a play down event can be deferred is 24 hours. You can adjust this via the following command:

Set-MailboxDatabaseCopy <database name\server> -ReplayLagMaxDelay:<value in the format of 00:00:00>

If you want to disable deferred play down, you can set the ReplayLagMaxDelay value to 00:00:00.

The following events are recorded in the Microsoft-Exchange-HighAvailability/Monitoring crimson channel when log replay is deferred or resumed:

• Event 750 – Replay Lag Manager requested activating replay lag delay (suspending log replay) for database copy '%1\%2' after a suppression interval of %4. Delay Reason: %6"
• Event 751 – Replay Lag Manager successfully activated replay lag delay (suspended log replay) for database copy '%1\%2'. Delay Reason: %4"
• Event 752 – Replay Lag Manager failed to activate replay lag delay (suspend log replay) for database copy '%1\%2'. Error: %4"
• Event 753 – Replay Lag Manager requested deactivating replay lag (resuming log replay) for database copy '%1\%2' after a suppression interval of %4. Reason: %5"
• Event 754 – Replay Lag Manager successfully deactivated replay lag (resumed log replay) for database copy '%1\%2'. Reason: %4
• Event 755 – Replay Lag Manager failed to deactivate replay lag (resume log replay) for database copy '%1\%2'. Error: %4
• Event 756 – Replay Lag Manager will attempt to deactivate replay lag (resume log replay) for database copy '%1\%2' because it has reached the maximum allowed lag duration. Detailed Reason: %5

The following events are recorded in the Microsoft-Exchange-HighAvailability/Operational crimson channel when log replay is deferred or resumed:

• Event 748 – Log Replay suspend/resume state for database '%1' has changed. (LastSuspendReason=%3, CurrentSuspendReason=%4, CurrentSuspendReasonMessage=%5)
• Event 2050 – Suspend log replay requested for database guid=%1, reason='%2'.
• Event 2051 – Suspend log replay for database guid=%1 succeeded.
• Event 2052 – Suspend log replay for database guid=%1 failed: %2.
• Event 2053 – Resume log replay requested for database guid=%1.
• Event 2054 – Resume log replay for database guid=%1 succeeded.
• Event 2055 – Resume log replay for database guid=%1 failed: %2.

Summary

The changes discussed above continue our work in improving the Preferred Architecture by ensuring that users have the best possible experience on the Exchange platform.

As always, we welcome your feedback.

Ross Smith IV
Principal Program Manager
Office 365 Customer Experience