How to configure Ex 2013 Throttling for Migration

Recently I came across a very good example where Ex 2013 throttling issues were interfering with the mailbox migration so wanted to share here. Partially it was due to MAgE agents not caching Autodiscover requests properly (QMM 8.11 fix is coming out soon around spring 2015, 8.10 is already patched). So what was happening, 40 agents were running with 5 threads each and all these 40×5=200 instances were hammering Ex 2013 with too many Autodiscover requests. Even 8 load balanced Ex 2013 boxes considered it too much and Ex 2013 is smart enough to recognize such “offenders”…. unless you raise the bar with the throttling policy.

Here are the error messages you may see in QMM MAgE Logs. The agents will still work and sync mailbox content, but it will be less efficient as part of its work cycle agent will spend dealing with errors until Ex 2013 unlocks the access, then locks it again for couple minutes.

3/18/2015 11:52:11 PM AutodiscoverResponse <Trace Tag=”AutodiscoverResponse” Tid=”32″ Time=”2015-03-18 23:52:11Z” Version=”15.00.0516.014″>
<s:Envelope xmlns:s=”https://schemas.xmlsoap.org/soap/envelope/” xmlns:a=”https://www.w3.org/2005/08/addressing”>
<s:Header>
<a:Action s:mustUnderstand=”1″>https://schemas.microsoft.com/exchange/2010/Autodiscover/Autodiscover/GetUserSettingsResponse</a:Action>
<h:ServerVersionInfo xmlns:h=”https://schemas.microsoft.com/exchange/2010/Autodiscover” xmlns:i=”https://www.w3.org/2001/XMLSchema-instance”>
<h:MajorVersion>15</h:MajorVersion>
<h:MinorVersion>0</h:MinorVersion>
<h:MajorBuildNumber>1044</h:MajorBuildNumber>
<h:MinorBuildNumber>21</h:MinorBuildNumber>
<h:Version>Exchange2013_SP1</h:Version>
</h:ServerVersionInfo>
</s:Header>
<s:Body>
<GetUserSettingsResponseMessage xmlns=”https://schemas.microsoft.com/exchange/2010/Autodiscover”>
<Response xmlns:i=”https://www.w3.org/2001/XMLSchema-instance”>
<ErrorCode>ServerBusy</ErrorCode>
<ErrorMessage>The server is too busy to process the request.</ErrorMessage>
<UserResponses />
</Response>
</GetUserSettingsResponseMessage>
</s:Body>
</s:Envelope>
</Trace>

Here is what you will see on Exchange 2013 Mailbox (! – not CAS) Servers:

a1sx2_Original1_2015-03-20_21-59-30

Log Name:      Application
Source:        MSExchange ADAccess
Date:          3/20/2015 2:15:26 PM
Event ID:      4023
Task Category: General
Level:         Error
Keywords:      Classic
User:          N/A
Computer:      EXCHANGE2013MBX.DOMAIN.COM
Description:
Process w3wp.exe (AutoDisc) (PID=19436). The budget for user ‘Sid~DOMAIN\SVC_QMM~Ews~false’ is locked out until 3/20/2015 6:20:26 PM.  Max Burst: 300000, Recharge Rate: 900000, CutoffBalance: -3000000
Event Xml:
<Event xmlns=”https://schemas.microsoft.com/win/2004/08/events/event”>
<System>
<Provider Name=”MSExchange ADAccess” />
<EventID Qualifiers=”49152″>4023</EventID>
<Level>2</Level>
<Task>1</Task>
<Keywords>0x80000000000000</Keywords>
<TimeCreated SystemTime=”2015-03-20T18:15:26.000000000Z” />
<EventRecordID>3474983</EventRecordID>
<Channel>Application</Channel>
<Computer>EXCHANGE2013MBX.DOMAIN.COM</Computer>
<Security />
</System>
<EventData>
<Data>w3wp.exe (AutoDisc)</Data>
<Data>19436</Data>
<Data>Sid~DOMAIN\SVC_QMM~Ews~false</Data>
<Data>3/20/2015 6:20:26 PM</Data>
<Data>300000</Data>
<Data>900000</Data>
<Data>-3000000</Data>
</EventData>
</Event>

And finally – how to properly configure throttling policy in Ex 2013 during migration. The way I am doing it below is not yet documented with the product, and possibly will not be updated because of Autodiscover requests caching that will be implemented soon. However I see no problems raising all EWS…. parameters for the throttling policy assigned to the migration account so here you go:

Set-ThrottlingPolicy MigrationPolicyUnlimited -PowerShellMaxConcurrency unlimited –RcaMaxConcurrency unlimited -RcaMaxBurst unlimited -RcaCutOffBalance unlimited -EwsMaxConcurrency unlimited -EwsMaxSubscriptions unlimited ––EWSMaxBurst unlimited –EWSRechargeRate unlimited –EWSCutOffBalance unlimited