Quantcast
Channel: You Had Me At EHLO…

New Outlook for iOS and Android App Configuration Policy Experience – General App Configuration

$
0
0

At Microsoft Ignite, Outlook for iOS and Android announced support for deploying managed device general app configuration settings for Office 365 mailboxes and on-premises mailboxes leveraging hybrid modern authentication. This capability leverages either the Managed App Configuration for iOS or the Android managed configurations to enable MDM solutions to push configuration settings.

Today, we are announcing the availability of new functionality within Intune that enables admins to easily deploy general app configuration to Outlook for iOS and Android via App configuration policies. This new functionality allows IT admins to configure the default behavior for several settings within Outlook for iOS and Android, such as Focused Inbox.

Note: For Outlook for iOS and Android to apply these settings, the app needs to be installed and managed by the Company Portal.

Figure 1: App Configuration Policy for Outlook for iOS on enrolled iOS devices from https://devicemanagement.microsoft.com. If you're in https://portal.azure.com, then you'll go to Intune -> Client apps -> App configuration policies and add a configuration policy. 

General App Configuration details

With this new policy experience, administrators can simply configure certain Outlook app settings’ default behavior and deploy them to their user’s enrolled mobile devices. For this first release, Outlook is supporting the following settings for configuration:

Setting Default app behavior Notes
Focused Inbox On
Require Biometrics to access the app Off This setting is only available for Outlook for iOS.

 

If using App Protection Policies, Microsoft recommends disabling this setting to prevent dual access prompts.

Save Contacts Off User must grant access to the native Contacts app for contact sync to occur.
External Recipients MailTip On
Block external images Off

 

As you may have noticed, settings that are security-related in nature have an additional option, Allow user to change setting. For these settings (Save Contacts, External recipients MailTip, Block external images, and Require Biometrics to access the app), administrators can prevent the user from changing the app’s configuration; in other words, the administrator’s configuration cannot be overridden.  Allow user to change setting does not change the app behavior. For example, if the admin enables Block external images and prevents user change, then by default external images will not be downloaded in messages; however, the user can manually download the images for that message body.

Note: The Allow user to change setting for Require Biometrics to access the app is currently only available as a configuration key. This will be addressed in a future Intune portal update. For more information regarding the configuration key, please see Deploy app config settings.

The following conditions apply with respect to Outlook’s behavior when implementing app configuration:

  • If the admin configures a setting with its default value, and the app is configured with the default, then the admin’s configuration doesn't have any effect. For example, if the admin sets External recipients MailTip=on, the default value is also on, so Outlook’s configuration doesn't change.
  • If the admin configures a setting with the non-default value and the app is configured with the default, then the admin’s configuration is applied. For example, the admin sets Focused Inbox=off, but app default is on, so Outlook’s configuration for Focused Inbox is off.
  • If the user has configured non-default value, but the admin has configured a default value and allows user choice, then we retain the user’s configured value. For example, the user has enabled contact sync, but the admin sets Save Contacts=off and allows user choice, so Outlook keeps contact sync on and does not break caller-ID for user.
  • If the admin disables user choice, then Outlook always enforces the admin defined configuration, regardless of the user's configuration or default app config. For example, the user has enabled contact sync, but the admin sets Save Contacts=off and disables user choice, so contact sync gets disabled and the user is prevented from enabling it.
  • If after the MDM configuration is applied, if the user changes the setting value to not match the admin desired value (and user choice is allowed), then the user’s configuration is retained. For example, block external images is off by default, admin set Block external images=on, but afterwards, user changes block external images back to off; in this scenario, block external images remains off the next time the policy is applied.

Users are alerted to configuration changes via a notification toast in the app:

Figure 1: Outlook for iOS and Android app config notification toast

This notification toast will automatically dismiss after 10 seconds. There are two scenarios where this notification toast will not appear:

  • If the app has previously shown the notification in the last hour.
  • If the app has been installed in less than 24 hours.

Save Contacts

The Save Contacts setting is a special case scenario because unlike the other settings, this setting requires user interaction – the user needs to grant Outlook permissions to access the native Contacts app and the data stored within. If the user does not grant access, then contact sync cannot be enabled.

Note: With Android Enterprise, administrators can configure the default permissions assigned to the managed app. Within the policy, you can define that Outlook for Android is granted READ_CONTACTS and WRITE_CONTACTS within the work profile; for more information on how to assign permissions, please see Add app configuration policies for managed Android devices. When assigning default permissions it is important to understand which Android Enterprise deployment models are in use, as the permissions may grant access to personal data.

The workflow for enabling Save Contacts is the same for new accounts and existing accounts.

  1. The user is notified that the administrator has enabled contact sync. In Outlook for iOS, the notification occurs within the app, whereas, in Outlook for Android, a persistent notification is delivered via the Android notification center.

    Figure 3: User notification regarding contact sync
  2. If the user taps on the notification, the user is prompted to grant access:

    Figure 4: User is prompted to grant access to native Contacts app
  3. If the user allows Outlook to access the native Contacts app, access is granted and contact sync will be enabled. If the user denies Outlook access to the native Contacts app, then the user is prompted to go into the OS settings and enable contact sync:

    Figure 5: User is prompted to enable contact sync in OS settings
  4. In the event the user denies Outlook access to the native Contacts app and dismisses the previous prompt, the user may later enable access by navigating to the account configuration within Outlook and tapping Open Settings:

    Figure 6: User can re-enable contact sync access in OS settings

Summary

We hope you enjoy this new policy experience available within the Intune portal for Outlook for iOS and Android. We'll continue to update the list of settings that can be managed via the MDM OS channel.

For more information on general app config with Outlook for iOS and Android, see Deploy app config settings. Up next is general app configuration for the without enrollment scenario. Stay tuned!

Ross Smith IV
Principal Program Manager
Customer Experience Engineering

 

Common questions

Q: What versions of Outlook for iOS and Android support general app configuration on enrolled devices?

Outlook for iOS 3.15.0 and Outlook for Android 3.0.30 and later support this functionality.

Q: Can I deploy general app config to Outlook for iOS and Android if the device is not enrolled?

Not at this time, but in the future, we plan to support this scenario for accounts that have an Intune App Protection Policy applied.

Q: What if I had already deployed the configuration keys manually in an App Configuration Policy; do I need to do anything?

No! The keys will be automatically consumed in the new policy experience.

Q: How do I create an App Configuration Policy for Outlook for iOS or Outlook for Android?

We’ll be updating Deploy app config settings to include the new policy experience, but you can also review Add app configuration policies for managed iOS devices and Add app configuration policies for managed Android devices.

Q: What if we are not using Intune to manage device enrollment, but instead are leveraging a third-party MDM solution?

Not to fear, we have you covered. These settings can be delivered via any MDM provider. For more information on the configuration keys you need to use, see Deploy app config settings.


Investigating TLS usage for SMTP in Exchange Online

$
0
0

Microsoft is committed to enforcing the best security for our services. As a result, TLS1.0, TLS1.1, and 3DES were deprecated in the Office 365 service.

While 3DES is currently in the process of being disabled, there is no date set for disabling TLS1.0 and TLS1.1. That said, we are working towards disabling these TLS versions for Exchange Online endpoints. Should TLS1.0 be compromised, we will have to act quickly to disable it in our service to protect our customers. In the case of SSL3.0, we disabled it in the service just over a month after the compromise was disclosed. Therefore, we urge you to be proactive by verifying TLS1.2 support for all of your email clients and servers as soon as possible.

For inbound and outbound connections with email servers and devices that are exposed to the internet, TLS1.0 usage is still around 5%. In most cases, TLS usage is optional for messages that are sent and received on the internet. There are certain scenarios where TLS is mandatory, and if TLS1.0 is turned off in Exchange Online, mail flow will be affected.

For example, over 10% of connections from customer on-premises email servers and devices still use TLS1.0. Even worse are the legacy SMTP Auth client submissions that are used by multi-function printers and applications that need to send email. For the SMTP Auth protocol, just less than 50% of connections are still using TLS1.0. These are likely old printers or legacy applications that either have not or cannot be updated to use TLS1.2.

To help you identify if your organization is contributing to those numbers, we have developed several reports for Exchange Online. You can use these reports to help determine which clients and servers are still using TLS1.0 and TLS1.1 to connect to the various email protocol endpoints in Exchange Online. These reports can be found in the Security and Compliance Center under the Mail Flow Dashboard.

Emails between your on-premises or partner email servers and Exchange Online

Third-party email servers sending and receiving email to and from our customers are normally beyond our control (or even the control of our customers). However, your on-premises or partner email servers are easily identified because their connections to and from Exchange Online use mail flow connectors. Exchange Online relies on successful TLS negotiations and certificates to identify and use the correct inbound connector. You can also configure outbound connectors to force the use of TLS. If a connector with forced TLS uses TLS1.0 today, messages will fail to send when TLS1.0 is disabled in Exchange Online.

To help identify servers that require updating to TLS1.2, we have developed the Connector Report, which is available in our Mail Flow Dashboard in the Security and Compliance Center. To access the report, click View Details and then the Connector Report link.

TLSreport1

The Connector Report allows you to review mail flow volume or TLS usage for a specific connector, or traffic to and from the internet that does not use a connector.  The numbers behind the charts are available in the Details Table. For detailed information on the messages involved (including if 3DES is being used), you can download the data using the Request report feature. From that data, you can identify the exact server or device, and you can attempt to upgrade the server or device to TLS1.2.

Email submitted using the legacy SMTP Auth client submission protocol

Email clients can submit email messages using several different protocols. The SMTP Auth protocol is a widely supported protocol that’s used primarily by devices and applications that send automated messages on behalf of customers. Examples include scanner to email devices, or applications that send out alerts or notifications. SMTP Auth is identified by its endpoint smtp.office365.com.

To protect against the disclosure of credentials, TLS is mandatory for SMTP Auth. This means that when TLS1.0 is disabled, no messages can be sent from devices or clients that do not support TLS1.2.

To help identify which of your devices and applications are still using TLS1.0, we have created the SMTP Auth Clients report. This report is available in the Mail Flow Dashboard where its widget displays the number of mailboxes that have used SMTP Auth in the last week. The report displays pivots for sending volume and TLS version usage. The details table provides the individual users or system accounts and their volume or TLS usage. You can also download the data using the Request report feature, which includes information about whether or not 3DES is being used.

TLSreport2

Email submitted using one of Microsoft's client submission protocols

The previously described reports apply to SMTP-related mail flow and submissions. For other protocols, a report is available on the Secure Score site. You can find if you have any TLS 1.0/1.1 and 3DES usage for Exchange Online by clicking Score Analyzer and scrolling to the Remove TLS dependencies tab.

If you want details on who is connecting using these weaker ciphers and protocols, click on the Update button and then Launch now on the flyout that appears.

This will take you to the Secure Trust Portal where you can download your TLS 1.0/1.1 and 3DES reports.

With these reports, you can now investigate the TLS usage in your Office 365 organization and take the necessary actions to avoid any mail flow disruptions in the future.

Sean Stevenson

Announcing the updated Exchange Deployment Assistant site

$
0
0

Today we’re announcing a new site for the Exchange Deployment Assistant – https://assistants.microsoft.com! Like the rest of the Exchange documentation, the Deployment Assistant has lived on TechNet since it was created. With the move towards the docs.microsoft.com platform, we needed to make a similar migration for the Deployment Assistant.

With the move to the new assistants.microsoft.com site, we’ve updated the Deployment Assistant to focus solely on on-premises deployments. For online and hybrid scenarios, we’ve had two different resources offering the same guidance: the EDA, and the Office 365 mail migration advisors. Rather than having the same guidance in two different places, we’re standardizing on the Office 365 mail migration wizards for online and hybrid deployments.

When you go to the new site, you’ll see two options:

eda

The first option, On-premises Exchange deployments, will go to the Deployment Assistant and all the on-premises deployment options you know and love today. You can select questions and answers to build checklists that help you deploy new Exchange 2013 or Exchange 2016 organizations, or upgrade to them from previous versions of Exchange. Exchange 2019 scenarios are coming soon!

The second option, Migrate Exchange to Office 365 will take you to the Office 365 mail migration advisors. The Office 365 mail migration advisors offer the best solutions for helping you migrate your organization to Office 365. These include staged and cut-over migrations and all flavors of hybrid migrations. They’ll help you decide which migration option is best for you and even make sure your Office 365 tenant is ready for the migration!

For those of you who are in the middle of an on-premises deployment or migration to Office 365, the existing Deployment Assistant will remain available until the end of April 2019. After that, and for everyone who’s looking to start a new deployment or migration, please start using the new Deployment Assistant at https://assistants.microsoft.com.

Finally, we want to host additional assistants at https://assistants.microsoft.com in the future, for Exchange and other products. What are some guided assistants you’d like to see? Let us know in the comments!

Thank you!

The Exchange Content Team

Mail flow insights (wave 2) will soon be available in O365 Security & Compliance Center

$
0
0

As you might have previously read, we released the first wave of mail flow insights last year (here is the original announcement).

Admins can use the mail flow dashboard in the Office 365 Security & Compliance Center to discover trends, insights, and take actions to fix issues related to mail flow in their Office 365 organization.

We're excited to announce that second wave of mail flow insights will soon be available in the Office 365 Security & Compliance Center. The new insights will be rolled out to customers who have opted in to Targeted Release, and will start to show up in the admin's mail flow dashboard at the beginning of April 2019.

We'll continue to create and refine mail flow insights to help improve productivity for admins, and we'll announce them as they become available.

You can see the details in this doc, but a quick summary is listed below.

Where to find mail flow insights?

  • If you are a global admin or an Exchange administrator, you can go to the Office 365 Security & Compliance Center at https://protection.office.com.
  • Expand Mail flow in the left hand nav, select Dashboard, you will see all insights on the right panel.

v2mailinsight1

As the doc we linked to earlier details, we're creating six new insights, reports and widgets available. Here are just a few examples of what you'll find in the mail flow insights dashboard.

Mail Flow Map

This report provides a visual map showing how mail flows through your Office 365 organization. You can use this information to learn patterns, identify anomalies, and fix issues as they arise.

v2mailinsight2

v2mailinsight3

Domain mail flow status

The Top domain mail flow status report gives you the current mail flow status for your organization's domains. This insight helps you identify and troubleshoot domains that are experiencing mail flow impacting issues (such as not receiving external email), domain expirations or domains with incorrect MX records.

v2mailinsight4

SMTP Auth client status

This report allows you to detect potentially compromised accounts due to the use of legacy (less secure) protocols. Clicking the widget will allow admins to see details of accounts that are still using the SMTP Basic Auth protocol, and will allow them to investigate potentially compromised accounts.

v2mailinsight5

v2mailinsight6

We really do encourage you to take advantage of mail flow insights in the Office 365 Security & Compliance Center and we hope you appreciate these additions. We will continue to improve the current insights as well as add new insights, and we're looking forward for your feedback!

Note that you can click the Feedback button at the bottom of the page to give feedback directly from the Security & Compliance Center:

Carolyn Liu

Exchange Team blog is moving to a new home soon!

$
0
0

EHLO!

As we hinted in our 15 year blogoversary post, this blog will be moving to a new home soon. Here is what you can expect:

  • Our new home will be Microsoft Tech Community, Blogs section (there will be “Exchange” in the list once all is said and done, with its own unique URL)
  • We have done a lot of cleanup of our 15 years of content already; generally speaking – if it is live on the blog today, it will migrate over. We did not try to delete ‘legacy’ content, rather get rid of stuff that is obviously not relevant anymore (like that Exchange 5.5 Webcast announcement from 14 years ago)
  • Yes, we will be migrating post comments over too; however, due to reasons… all of the comments will migrate over as anonymous; we felt there was still value there to keep the discussion present even though it might not be immediately obvious who was asking and replying exactly. The context helps in many cases, and good advice is good advice so…
  • We will work to make sure all of individual current post URLs redirect to migrated posts in their new location
  • We will work to redirect short URLs that go to our blog today, like http://aka.ms/ehlo
  • All of this is going to happen within next 3-6 weeks. Or earlier. Or later. Basically: “When it’s ready.”

What can you do to prepare?

If you want to keep being engaged with us and want to keep commenting to our posts (and continue the discussion that Exchange community is known for), you will need to register with Tech Community site. We are not doing any kind of migration of user accounts.

When we will be in the finalization stage, there is likely going to be some time when we will take a ‘snapshot’ and migrate that, and any comments etc. made after that will be lost once we move over (as they will not be in migration set). We will give you a heads up when this is about to happen. If unexpected things happen and you start getting errors temporarily, now you know what’s up!

The Exchange Team Blog Team

Exchange Online – Modern Authentication and Conditional Access Updates

$
0
0

We’re constantly improving the security of Office 365 products and services. Modern Authentication and Conditional Access are two of the best ways of ensuring that your clients can take advantage of authentication features like multi-factor authentication (MFA), third-party SAML identity providers, and are implementing automated access control decisions for accessing your cloud apps based on conditions.

Firstly, here’s some news about Modern Authentication. As you might already know, all new Office 365 tenants created on or after August 1, 2017 have Modern Authentication enabled by default in Exchange Online for all clients.

Today, we’re announcing that Modern Authentication will soon be enabled for the Windows Outlook client and Skype for Business client in all managed (non-federated) tenants that were created before to August 1, 2017. Those tenants already have Modern Authentication enabled for Outlook mobile, Outlook for Mac and Outlook on the Web, so there are no changes to any of those clients.

What does it mean to be a ‘managed tenant’?

If you use Password Hash Sync, Pass-Through Authentication, or you create, manage and authenticate your user identities directly in the cloud, your tenant is considered a ‘managed tenant’ – and this change affects you.

If your still create, manage and authenticate your identities in your on-premises Active Directory, and you use ADFS or some other 3rd party iDP to authenticate your users – your tenant will not be affected by this change.

Will my user experience be different?

This change affects the dialog users will see when requesting their credentials.

They used to see the following prompt (the exact dialog depends upon the OS of the client, but this should be similar enough to help you identify it):

MApost1

Now they will see the following prompt:

MApost2

How does this change authentication?

From the user’s perspective, it’s just a dialog change. From a security perspective, the client is now using OAuth (not Basic Auth) to authenticate.

What’s better about that? Why do I care?

Switching to Modern Authentication (even if it’s used just for username and password) is more secure than using Basic Auth. Modern Authentication is not subject to credential capture and re-use, credentials are not stored on the client device, it ensures users re-authenticate when something about their connection or state changes, and it makes adding MFA simple.

What do I need to do as an Admin?

Nothing. Nothing at all, well except perhaps one thing: help your users understand that this new dialog means their connection to Office 365 is even more secure than it was before. Feel free to take the credit for that; tell them you changed it to increase their security; we don’t mind.

The next thing to do is to start thinking about enabling MFA and Conditional Access, to make those connections even more secure. Here’s a great place to start finding out more.

Speaking of Conditional Access, that leads us to the next thing we wanted to announce: we’re making some changes there too, specifically related to Exchange ActiveSync (EAS).

We’re making a change to ensure that EAS connections will be evaluated against previously unsupported conditions within Conditional Access (CA).

As you might know, many conditions that are available in CA policies have not been supported for EAS. These include country, named locations, sign-in risk, and device platform. Currently, if you include any of these conditions in a policy that targets EAS, that condition is always enforced. For example, a policy to require a compliant device outside of the corporate network would always apply (independent of the user’s location).

The below shows how the admin would enable the client app condition used to target CA policy to EAS clients.

MApost3

The change we have made ensures that CA policy applied to EAS correctly honors previously configured conditions. You may see some cases where EAS may begin to work where it was previously blocked. So, if you have CA policies today that block EAS traffic because a condition is not supported, we advise you inspect and remove any of the unsupported conditions from policy.

For example, suppose you previously configured the following policy: “Block all EAS traffic from French Guyana”. Today all EAS traffic is blocked. If you are relying on a rule like that to block all EAS traffic, you need to re-think your strategy.

With the change we are making, only the EAS traffic from French Guyana will be blocked. We’re sure that you find this behavior more logical, but we wanted to make sure you were aware of the change.

So, it’s worth checking your existing CA policies to make sure you don’t have rules that might be affected by this change.

Other than this, we don’t expect any other change in behavior: EAS clients should still receive quarantine email when they don’t meet the CA policy requirements; otherwise they will get email access just as they do today.

We really do treat the security of our service and the protection of your data as our primary concern.

Please leave any comments or feedback, and thanks for reading!

The Exchange Team

Introduction to Public Folder Hierarchy Sync

$
0
0

Introduction

The purpose of this article is to provide high level overview of hierarchy synchronization process in public folders and provide some troubleshooting steps to monitor and troubleshoot hierarchy related issues. The information presented here applies to public folder deployment in Exchange on-premises as well as Exchange Online, with any differences called out.

Before we get started, these are some of the terms used in public folder world:

Term Definition
Modern public folders Public folder architecture and deployment in Exchange Online and Exchange on-premises versions at or newer than Exchange Server 2013. In this architecture, public folders are stored in specialized mailboxes, called public folder mailboxes.
Public folder hierarchy The logical structure/skeleton of public folders and associated properties as well as permissions.
Public folder mailbox A special type of mailbox that is used to store public folder content and public folder hierarchy for modern public folders.
Public folder content The actual data stored within public folders.
Primary hierarchy public folder mailbox The public folder mailbox that hosts writable copy of the public folder hierarchy. The first public folder mailbox created in an Exchange Organization is the primary hierarchy mailbox. There is currently no supported way to transfer this functionality to another mailbox.
Secondary hierarchy public folder mailbox All other public folder mailboxes in an Exchange organization, except the primary hierarchy, which store read-only copy of the public folder hierarchy.
Hierarchy synchronization The process of copying the public folder hierarchy between public folder mailboxes. The hierarchy replication is always from primary hierarchy public folder mailbox (which contains the only writeable copy of the hierarchy) to secondary hierarchy mailboxes.
Public folder content mailbox The public folder mailbox storing the actual content of a public folder. In modern public folders, there is only one copy of content.
Hierarchy mailbox Any public folder mailbox that is enabled to serve the hierarchy information to clients is referred as a hierarchy mailbox.
Full synchronization The process of synchronizing entire hierarchy to secondary public folder mailboxes. Note that this is a hierarchy only synchronization.
Incremental synchronization The process in which only changes made in hierarchy, after last sync, are synchronized to secondary hierarchy PF mailboxes.

Public folder hierarchy sync

In simple terms, the public folder hierarchy is the skeleton or logical structure in which the permissions and other properties of public folders are arranged. The hierarchy makes it possible for clients to determine which PF mailbox stores the actual content of the public folder. The hierarchy is synchronized across all public folder mailboxes, to ensure a single PF mailbox is not burdened with serving the hierarchy to all clients.

Frequent synchronization of the hierarchy is critical to ensure end users always get an up to date view of the hierarchy.

Here’s what can happen when the hierarchy is not in sync. Public folder mailbox EPF1 is the primary hierarchy mailbox and has two folders in the hierarchy. A new PF mailbox EPF2 is created. EPF2 won’t show the complete public folder structure until the hierarchy is synced:

pfhierarchy1

Let’s discuss how the public folder synchronization works and the components involved.

Pull mode

In pull mode, hierarchy replication is driven by secondary hierarchy mailboxes. At regular interval, the secondary hierarchy mailbox connects to the primary hierarchy mailbox to pull the changes in hierarchy. This process is accomplished with the help of the ServiceHost process (running on each server) and the MRS proxy service (running on the server hosting the primary mailbox). The frequency of connection depends on whether clients were connected to the mailbox or not.

This workflow is executed with the following frequencies:

Mailbox Type Frequency Notes
Primary Never Primary is the master of the hierarchy; it does not pull from anywhere else
Secondary with users connected 15 minutes At least once every 15 minutes as long as there are client connections to the mailbox
Secondary with no user connections 24 hours All secondary mailboxes are synced once daily.

Additionally, the following situations also trigger sync operations between a secondary mailbox and primary:

  1. A user connected to a secondary mailbox performs a hierarchy update / change: The operation is performed in the primary hierarchy mailbox and then immediately replicated to the PF mailbox that is serving hierarchy for the user who made the change.
  2. An update is done in a folder whose content is hosted in a mailbox other than the primary mailbox: The operation is first applied in primary (all write hierarchy client operations get proxied there) and then replicated immediately to the mailbox hosting the affected folder.
  3. An admin user invokes the Update-PublicFolderMailbox with the InvokeSynchronizer parameter.
  4. A new mailbox is created by Migration or AutoSplit (in Exchange Online only): These processes will cause a full hierarchy sync to be executed in the new mailbox.

The process described above was the default model for full as well as incremental hierarchy synchronization in the early days of modern public folders and served well as long as the number of public folder mailboxes was not high. However, there were some limitations with this model:

  • Secondary mailboxes had no way to know if there had been a change in hierarchy. In other words, secondary mailboxes would keep connecting to the primary at defined intervals, even if there was no change in hierarchy.
  • More secondary mailboxes meant more sync jobs, thus more connections competing for the limited resources in the primary mailbox.
  • Larger hierarchies mean more data during sync (larger sync states and more folders to transfer) which meant longer execution times and higher resource consumption (memory, IO, CPU and network).
  • As the size of the user base increases it means more hierarchy mailboxes (and consequently a higher number and frequency of sync jobs) and this usually leads to more hierarchy churn (thus requiring longer sync jobs as more data is processed).

To address these problems, a new “push mode” of hierarchy sync was introduced.

Push mode

The push mode of hierarchy sync is driven by primary PF mailbox, mainly for the incremental synchronization of hierarchy. This change was introduced in Exchange Server 2016 CU2 on-premises and in Exchange Online around the same time.

In push mode, the primary PF mailbox proactively:

  • Notifies secondary mailboxes when there have been changes.
  • Sends any data the secondary mailbox may need to apply the recent changes, limiting the need for it to contact primary.

Pull mode is still used for:

  • Initial sync of newly created PF mailboxes
  • Full sync requests triggered by an administrator
  • Fallback mechanism in case of push notification being lost, or when mailboxes become out of sync

In push mode, the primary PF mailbox drives the incremental hierarchy sync jobs, using transport to send email messages to a hidden distribution list containing the different public folder mailboxes present in the hierarchy.

Some key properties of this distribution list:

  • It is a hidden Dynamic DL.
  • It has ShowInAddressList set to false.
  • Its identity will be persisted in the organization object so mailboxes can refer to it.
  • You should normally find only one such dynamic distribution group created in an Exchange environment.

The hidden DL created for sending push notification messages cannot be seen in Exchange Online but you can use the following command to view the hidden DL in Exchange on-premises:

Get-DynamicDistributionGroup -IncludeSystemObjects pub*

pfhierarchy2

pfhierarchy3

Secondary mailboxes will use both modes of sync, pull mode for initial full sync and push mode for incremental hierarchy sync, depending on the sync state they are in. The first sync is performed using pull mode, after that, they will not contact primary mailbox repeatedly for incremental syncs. Instead, they wait for the hierarchy sync notification email and apply the hierarchy changes once the email is received. If the secondary hierarchy mailbox doesn’t receive a push notification message within 10 minutes, it will fall back to pull mode and perform the incremental sync by contacting the Primary mailbox.

Secondary mailboxes will also use the pull mode if an administrator triggers hierarchy sync.

Hierarchy mailbox assignment

Exchange by default automatically assigns a default public folder mailbox on all user mailboxes, load balancing between all available and assignable public folder mailboxes.

A public folder mailbox having following properties is considered available for automatic assignment. In the following example, both public folder mailboxes are considered available for automatic assignment:

pfhierarchy4

An admin can exclude specific PF mailbox from being automatically assigned by configuring IsExcludedFromServingHierarchy to $True

Example:

Set-Mailbox -PublicFolder epf1 -IsExcludedFromServingHierarchy $True

An admin can override the system assignment by using this command:

Set-Mailbox <username> -DefaultPublicFolderMailbox <PFMailboxName>

Example:

Set-Mailbox cloud1 -DefaultPublicFolderMailbox epf1

The public folder mailbox serving hierarchy for the user can be found using this command:

Get-Mailbox |ft name,DefaultPublicFolderMailbox,EffectivePublicFolderMailbox

The admin assigned mailboxes appear under “DefaultPublicFolderMailbox”; whereas the system assigned PF mailbox will be displayed under “EffectivePublicFolderMailbox” and DefaultPublicFolderMailbox will be blank.

pfhierarchy5

For larger deployments, avoid manually assigning DefaultPublicFolderMailbox on users, as it may overload the assigned PF mailbox with lots of concurrent connections. As we learned earlier, the system assigns DefaultPublicFolderMailbox on users in such a way that connections are load balanced between public folder mailboxes available for serving hierarchy. Follow the best practices guidelines here for public folder mailbox placement and assignment.

Making sure hierarchy is healthy

Here are some tips and tricks to help keep your public folder hierarchy in top shape.

Full hierarchy sync

Admins can use the following command to ensure that the public folder mailbox has received the initial full sync of hierarchy and confirm that the PF mailbox is ready to serve hierarchy. The value of True returned here indicates the mailbox has received its first full sync of hierarchy

Get-Mailbox -PublicFolder |ft Name,IsHierarchyReady

pfhierarchy6

Diagnosing hierarchy sync issues

If you suspect a specific PF mailbox is not receiving hierarchy (perhaps users tell you they can’t see a folder they know to exist), you can use the following command to get diagnostic information about hierarchy:

Get-PublicFolderMailboxDiagnostics <pfmailboxname_notreceiving_hierarchy> -IncludeHierarchyInfo

Here are some other ways to view data;

To compare the hierarchy between PF mailboxes

$P=Get-PublicFolderMailboxDiagnostics <Primary_pfmailboxname> -IncludeHierarchyInfo
$S=Get-PublicFolderMailboxDiagnostics <pfmailboxname_notreceiving_hierarchy> -IncludeHierarchyInfo

Then, you can check the output of “HierarchyInfo” from both the mailboxes and compare:

$p.HierarchyInfo
$s.HierarchyInfo

pfhierarchy7

If you find the hierarchy information is not the same, you can use following command to view the time of the last sync:

$S.AssistantInfo.LastAttemptedSyncTime.LocalTime

pfhierarchy8

This command tells you the last time sync failed; a gibberish value (that’s a technical term now!) indicates sync has never failed:

$s.AssistantInfo.LastFailedSyncTime.LocalTime

pfhierarchy9

The following command will give you a detailed failure message from the last sync failure, a blank output indicates sync has never failed:

$s.AssistantInfo.LastSyncFailure

You can explore the other values of AssistantInfo, SyncInfo & HierarchyInfo blocks.

In case the need arises to contact Microsoft Support for assistance, you should export the report to XML format and send it along.

Get-PublicFolderMailboxDiagnostics <pf mailbox failing to sync> -IncludeHierarchyInfo |Export-Clixml epf2.xml

Other useful information

The following are some things to know related to the hierarchy sync of public folders:

Push notification emails and journaling

Since push notifications are an email message delivered using transport they will be included in journaling if enabled in an organization. In such a scenario, you can use scoped journaling and exclude the SMTP address of primary public folder mailbox from the journaling rule (as that is a sender of push notification messages).

Public folder mailbox is a top sender

You may observe that the public folder mailbox shows in the “Top Sender and recipient report” in the Security & Compliance Center.

pfhierarchy10

This is currently normal and not a sign of a problem, because the primary public folder mailbox sends a large number of push notification emails to secondary mailboxes via transport. The next version of the report will exclude the public folder hierarchy sync emails.

Multiple dynamic DL’s created

As discussed earlier, push notification emails are sent to an Exchange-managed dynamic distribution group. Normally, only one dynamic distribution group is enough for the push notification emails. However, in environments that have multiple domain controllers within a single AD site with AD replication issues between the DCs, multiple dynamic distribution groups may get created unexpectedly. The issue happens because of a race condition and time lag between AD replication between DC’s in same site. Please follow steps in this KB to resolve this.

Conclusion

We hope this gives you better understanding of modern public folder hierarchy and some tools you can use to perform the troubleshooting. We would love to hear any feedback you may have on the topic.

The Public Folder Team

Microsoft Hybrid Agent Preview Update

$
0
0

As you are aware, we released the Microsoft Hybrid Agent for Exchange Server as a preview on Feb. 5th, 2019. The blog post for that release is located here.

We have been busy working on a few improvements for the Hybrid Agent over that past two months and wanted to let you know of some new features now available in the latest build of the Hybrid Configuration Wizard.

The three enhancements for the Hybrid Agent are:

  1. Multi Agent installation and configuration
  2. Agent status views
  3. Registration/Usage of a load balancer for the internal URL instead of a specific Exchange Client Access Server

Multi Agent Deployment

Option 1 – Use the Hybrid Configuration Wizard to install additional agents

For existing Hybrid Agent preview customers who would like to install additional Hybrid Agents for redundancy, simply download the latest version of the Hybrid Configuration Wizard (HCW) and open the application on the machine where you would like to install an additional Hybrid Agent.

  1. Like previous HCW runs, start the application, select Next.
  2. Select a desired server to execute against, select Next.
  3. Provide credentials to sign into your Office 365 tenant, select Next.
  4. The HCW will gather configuration information, select Next when complete.
  5. Select the default option provided for either Full or Minimal, select Next.
  6. Select Exchange Modern Hybrid Topology, Next.
  7. Agree to the Terms, Next.
  8. A new page will be shown that will provide you with the status of your existing / previously installed agent. Make sure the status of the existing agent is accurate before proceeding to the next step. Select Install and additional agent option, Next.

Example:

HyAgentUpd1

The HCW will install the additional Hybrid Agent. When the installation is complete, you can open the Microsoft Windows Services console from the machine and verify the service / agent is installed and running (look for Microsoft Hybrid Service - mshybridsvc). At that point, you can either re-run HCW if you wish to make further changes to your hybrid config, or simply cancel the wizard.

You can repeat this step on each machine where you would like an additional Hybrid Agent installed.

Option 2 – Manually download & install additional agents

A second option for installing additional agents is outside the HCW itself and is done by downloading and manually installing the agent on the desired machine.

  1. Go to https://aka.ms/hybridagentinstaller.
  2. Save the MSHybridService.msi to a location on your machine.
  3. From that machine, open a Windows Command console as Administrator and run:
  4. Msiexec /i MSHybridService.msi to install the hybrid agent. You will be prompted for your tenant Global Admin credentials.
  5. After the installation is complete, you can open the Microsoft Windows Services console from the machine and verify the service / agent is installed and running.

You can repeat this step on each machine where you would like an additional Hybrid Agent installed.

Checking the Status of Your Hybrid Agents

Option 1 – Get status via the Hybrid Configuration Wizard (HCW)

  1. Start the HCW application, select Next.
  2. Select a server in your Exchange org, select Next.
  3. Provide credentials to sign into your Office 365 tenant, select Next.
  4. The HCW will gather configuration information, select Next when complete.
  5. Select default option provided for either Full or Minimal, select Next.
  6. Select Exchange Modern Hybrid Topology, Next.
  7. Agree to the Terms, Next.
  8. A new page will be shown that will provide you with the status of your existing installed agents.

HyAgentUpd2

Cancel the HCW app when you are done.

Option 2 – Get status via the Hybrid Management PowerShell Module

With each installation of the Hybrid Agent, we install the Hybrid Management PowerShell module in the directory …\Program Files\Microsoft Hybrid Service\ on the machine where the agent is installed. By default, this module is not imported and so you will need to import it before you can use it. This module also requires the Azure module for PowerShell if not already installed. Please install the PackageManagement modules first and then see this article for how to install the Azure module.

To import the Hybrid Management module, run the following from a Windows PowerShell prompt as Administrator:

Import-module .\HybridManagement.psm1

After that you can run:

Get-HybridAgent -credential (get-credential) to view agent status:

HyAgentUpd3

Note: the ‘id’ value provided in the above snip is the agent identity and not your unique tenant guid assigned to the route.

Directing your Hybrid Agent(s) to the load balancer instead of a specific server

If you would like your Hybrid Agent(s) to direct requests to your load balancer instead of a specific Exchange Client Access Server, this can be done with the Hybrid Management PowerShell module. In preview, the Hybrid Agent supports routing requests to the load balancer for Exchange Server 2013 – 2019 Client Access Servers. You cannot use this if you only have Exchange Server 2010 CAS deployed.

Follow the steps above to import the Hybrid Management module for PowerShell.

From PowerShell, then simply use the Update-HybridApplication cmdlet to change the value of the internalURL (targetURI parameter) from a specific server to your load balancer endpoint.

Before running this cmdlet you will need to get the unique guid value (e.g. 12345678-1234-1234-1234-1111111111111) of your tenant’s endpoint from either your MRS configuration or the Organization Relationship object (TargetSharingEPR). Example:

PS C:\PowerShell> Get-OrganizationRelationship ((Get-OnPremisesOrganization).OrganizationRelationship) | Select-Object TargetSharingEpr
TargetSharingEpr
----------------
https://e399b770-3b66-407b-a71a-96ef80a67714.resource.mailboxmigration.his.msappproxy.net/EWS/Exchange.asmx

Or

PS C:\PowerShell> Get-MigrationEndpoint "Hybrid Migration Endpoint - EWS (Default Web Site)" | Select-Object RemoteServer
RemoteServer
------------
e399b770-3b66-407b-a71a-96ef80a67714.resource.mailboxmigration.his.msappproxy.net

To be clear, the ID parameter required in this cmdlet is not the agent ID, but the guid listed in your custom registered endpoint retrieved using the method above.

Example:

HyAgentUpd4

We really hope you are enjoying and benefitting from using the Hybrid Agent. We see a lot of usage and we’re working hard to improve it all the time.

Thank you!

Exchange Hybrid Team


Introducing the new migration experience from Google G Suite

$
0
0

Several weeks ago we added a new Microsoft 365 Roadmap item announcing our intent to add ability to migrate Google G Suite calendars and contacts to the ability to migrate mail to Office 365 using our native migration tools.  We're excited to say that this functionality has started rolling out!  You can expect to see the new features light up for your tenant in the coming weeks.

Since this is a new flavor of migration to Office 365, let's take a look at what is now becoming available and answer some frequently asked questions about it.  Before diving into any migration project, it is important to answer some basic questions about it:

  1. What is the size of the organization you are trying to migrate?
  2. Based on your business requirements, would you like to migrate using a "cutover" (everyone at once) or "staged" (longer-term coexistence) approach?
  3. What is the (approximate) average mailbox size that you'd be migrating?
  4. What kind of data do you need to migrate - like mail, calendar and contacts?
  5. Given the answers to the above questions, does this migration tool fit my needs?

Answering these questions will give you some idea of the scope of your project. Then you just need to find out how to get started. And you can read all about that here.

Before you actually start clicking though, let's cover some of the terms you'll need to understand and an overview of how the new G Suite migration flow will work:

Intro to Mailbox Replication Service (MRS)

Microsoft Exchange Mailbox Replication Service (MRS) is a component that handles mailbox import, export, migration and restoration for Exchange and Office 365.  Migrations are managed using individual requests.  MRS is the principal mechanism used to move mailbox data from one place to another.  In context of moving data from G Suite to Office 365 MRS is used to move mailbox data, including messages, contacts, and calendar items.  Each mailbox being migrated from G Suite to Office 365 has its own request that will be processed by MRS.

What is a migration batch?

For better usability and scheduling at scale, another component called the Migration Service provides the ability to submit migration requests for batches of users. Behind the scenes, the Migration Service uses the Mailbox Replication Service to manage the per-mailbox requests.  Grouping a set of users into a batch is primarily for management purposes and does not impact the speed or throughput of your migrations based on batch size.  For your migration, you could choose to have one batch of all users or split the users into multiple batches - the choice is yours.

Give me an overview; how do migration batches work?

MigBatchesflow

How is mail data migrated?

Mail data is currently migrated using IMAP protocol.  Authentication happens using domain-wide delegated access using a Service Account that is under your control.

How fast can I migrate my data?

For mail data there is a throughput limitation of 2 GB per mailbox per day. This limit is enforced by G Suite.

For contacts and calendar data, this completely depends on the quota restrictions for your tenant's service account on the Google G Suite side (because a different protocol is used to migrate calendars and contacts; please see documentation).

I reached my 2GB limit for the day (for mail) - will the migration continue the day after?

Yes - it will automatically continue once there is capacity to migrate more data, until the 2GB/day quota is reached.

What data will NOT be migrated over?

  • Mail: Vacation Settings or Automatic reply settings as well as Filters or Rules
  • Meeting Rooms: Room bookings
  • Calendar: Shared calendars, cloud attachments, Google Hangout links and event colors
  • Contacts: A maximum of three email addresses per contact are migrated over
  • Contacts: Gmail tags, contact URLs and custom tags are not migrated over

Will this also support migrating data from Google Vault to Office 365?

No.

What is the smallest batch size and is there an optimal batch size?

A migration batch can have a single user, with no current upper limit.  There is no magic number for the best migration speed or efficiency.  We recommend creating batches per-department or another logical grouping for your organization.

What is the size of the largest email I can migrate from G Suite to Office 365?

The limitation is based on the transport configuration for your organization.  The default limit is 35MB. Please see this article on how this limit can be increased.

I tried migrating and saw mention of bad items; what are those?

Bad items are data conversion failures that may be encountered during the syncing phase.  Should you encounter any bad items, you can see these in the migration report on the user and batch levels.

How do I track progress of migrations?

You can view the progress and reports via the Exchange Admin Center or use the Get-MigrationBatch PowerShell cmdlet.

Mail on Office 365 doesn't support labels.  How is mail translated to folders?

In order to provide the best mail experience in Outlook as well as account for the Focused/Other view, we translate most labels to folders.  Mail with the 'Starred' label will become flagged.  Mail with the 'Important' label will influence its Focused/Other designation.  Mail with other labels will be copied into a folder with the corresponding label name, and mail with no other labels will be moved to the Archive folder.  This means that some items that had multiple flags in G Suite will appear in multiple folders in Office 365.

I started off a batch in EAC, why doesn't my migration make progress?

Please make sure that you went through all of the pre-requisite steps on the G Suite side and have provisioned users on the O365 side.  Also check the per-user error messages if you see users with a status of 'Failed'.

My messages have completed copying but my migration is stuck at 95%. Help!

This could be expected.  Check the status of migration batch; if it reached the 'Synced' status, you can choose to complete the migration batch.

What does completion do?

For G Suite migrations, there's an option to complete a batch.  During completion, an additional incremental sync of data is performed, followed by a "switchover".  During switchover, a forwarding address is applied to the source G Suite mailbox to forward emails to the configured Target Delivery Domain.  Also, any forwarding address applied to the target O365 user is removed.

Do I really need to "Complete" my batch?

Depends - If you're taking the cutover migration route, you don't need to complete your batch(es). If you're taking the staged migration route (i.e., migrating a subset of users sequentially over a longer period of time), you'd want to complete certain batches if you'd like the automatic mail forwarding rules to reverse (Forward email from Gmail to Exchange Online instead of Exchange Online to Gmail).

How can I tell if this feature is available in my tenant?

As an Admin, you will see the G Suite (Gmail) migration option under the Migration tab in the Exchange Admin Center (EAC) or will have access to the -Gmail parameter for New-MigrationBatch CMDlet.

We really hope you enjoy using this new set of tools and it makes your migration from G Suite to Office 365 even easier. We really want to hear your feedback, so please do leave us a comment, and feel free to ask any questions here too. We have plans to make this migration toolset even better over time, so keep an eye on the blog.

The Exchange Migration Team

The 100 migration batches limit and how to not run into it

$
0
0

As you might or might not know, in Exchange Online we have an upper limit of 100 migration batches. You can see this limit for MaxNumberOfBatches in the Exchange Online PowerShell Get-MigrationConfig:

100batches

(To understand the MaxConcurrentMigrations limit of 300, you can check this blog post written by one of our migration experts, Brad Hughes.)

100 batches created at the same time should be enough for all O365 multi-tenant customers. But, as we have seen in support, sometimes there are support tickets opened because customers would see the error that says: “The maximum number of migration batches is already running. Please remove a batch before you add another one”.

The main reason why this number of batches limit is in place is that having more than 100 batches causes performance problems and could cause outages for other customers in the multi-tenant system. Additional reading on resourcing for mailbox moves can be found here.

Let’s say that you want to migrate 50,000 mailboxes to Exchange Online: you could create 50 batches with 1000 users each, and you would still have 50 batches available.

If you need to complete all the migrations of users from Batch_1 at the same time, you can easily set -CompleteAfter at the migration batch level. Once this batch is completed, you could delete it to create room for another one.

In real world, though, we have seen that our customers rarely want to complete large amount of mailbox migrations at the same time. This is usually the reason why large migration batches are not created but rather smaller batches are preferred. That way, all the migrated users from one migration batch at a time can be completed together.

In this scenario, our recommendation is to have large batches for initial sync and if you need to complete the migration for users at a separate date, then you would complete the migration for groups of users in smaller batches, as needed. So how would you create small completion batches for already synced users from larger synced batches?

Suppose you want to complete migration of users A, B and C, who reached Synced status and they are all contained in a large batch of 1000 users called Batch_1.

To do this, first ensure that you have fewer than 100 batches created at the moment:

(Get-MigrationBatch).count

Then, proceed with PowerShell commands to get the users from the larger batch Batch_1 into the new smaller batch CompletionABC. Then complete this smaller batch and optionally, remove the completed migration batch:

New-MigrationBatch -Name CompletionABC -UserIds EmailAddressOfUserA, EmailAddressOfUserB, EmailAddressOfUserC
Complete-MigrationBatch CompletionABC
Remove-MigrationBatch CompletionABC

Another method we sometimes see people using is setting -CompleteAfter on the move requests directly using Set-MoveRequest.  We don't recommend this method because the Migration Service may overwrite the per-request CompleteAfter setting with the one from the batch without notice. This method of individually completing the move requests should only be done when you’ve started the migration through PowerShell, using New-MoveRequest cmdlet, without creating migration batches to contain those move requests.

I hope this post gives you a better idea of how to manage migration batches with many users being migrated and not have to run into the limit of number of batches.

See you in the cloud!

Special thanks to Brad Hughes and Nino Bilic who contributed to this blog post and their precious help in my daily support job.

Mirela Buruiana





Latest Images