Tuesday, August 15, 2017

Using Dynamics365 Customer Engagement admin API with PowerShell, part1

Last week Microsoft released a new API for administering MSDYN365 Customer Engagement instances.
The API is REST based and easy to use, but as it is completely new it only supports OAuth authentication (which means no more simple cookie auth).
There is a sample included which does everything you need to get it working, but I made a few modifications to the AuthenticationHelper class to reuse it both for the MSDYN365 Customer Engagement API as well as the admin api.

So what I'm going going to do for this series is show you how to register an Azure Application and host this yourself (or you can just check out my github repo for the Dynamics365-PoSh)

Also, check out part2 here and part3 here

Create a new class project and scaffolding a commandlet

The first thing we'll do is to create a new class project for .Net Framework in Visual Studio (download link). Next, add the following NuGet packages:

Next up we start by scaffolding our initial class. Rename it to GetDynamicsInstances.cs, then inherit from the Cmdlet class. Decorate your class with cmdlet specifications, and create an overridden ProcessRecord method

[Cmdlet(VerbsCommon.Get, "DynamicsInstances")]
public class GetDynamicsInstances : PSCmdlet
{
    [Parameter(Mandatory = true)]
    [ValidateSet("NorthAmerica", "SouthAmerica", "Canada", "EMEA", "APAC", "Oceania", "Japan", "India", "NorthAmerica2", "UnitedKingdom", IgnoreCase = true)]
    public string Location;

    protected override void ProcessRecord()
    {
        base.ProcessRecord();
    }
}

With this we have a method that takes in a verifiable location. We can now compile, open up a PowerShell session, and import the compiled DLL as a module using the following line

Import-Module .\MSDYN365AdminApiAndMore.dll

When we try to run our Get-DynamicsInstances commandlet we can tab through the predefined set of locations.

Adding URL generator and base method for authentication

I've created a folder named Helpers, and added a class named UrlFactory. I've made the class static, and added a public enum to the end of the file to prevent usage of magic variables.

public enum DataCenterLocations
{
    NorthAmerica = 1,
    SouthAmerica = 2,
    Canada = 3,
    EMEA = 4,
    APAC = 5,
    Oceania = 6,
    Japan = 7,
    India = 8,
    NorthAmerica2 = 9,
    UnitedKingdom = 11
}

Next up is adding a method for generating a URL to use for the admin API. We'll take a location enum and an operation string as input. In addition, we'll add a static string for the URL format which we'll use to return the complete URI.

public static string BaseUrl = "https://{0}.crm{1}.dynamics.com{2}";
public static Uri GetUrl(string subdomain, DataCenterLocations location, string resource = "")
{
    if (location == DataCenterLocations.NorthAmerica)
    {
        return new Uri(
            string.Format(BaseUrl, subdomain, "", resource)
            );
    }
    else
    {
        return new Uri(
            string.Format(BaseUrl, subdomain, (int)location, resource)
            );
    }
}

This allows us to call the GetUrl method with only a location and the resource we want to call. In case of north america there is no number appended to the crm subdomain, so we're filling in blank there.
Next we'll create a new helper name AuthenticationHelper. This will take care of the authentication for us, and it is based on the sample in the admin API docs.
For now, we'll just add a public constructor which takes in the server url, and sets a private string value to the authority (strips the path away from the Uri).

public class AuthenticationHelper
{
    private string _endpoint = null;
    public AuthenticationHelper(Uri endpoint)
    {
        _endpoint = endpoint.GetLeftPart(UriPartial.Authority);
    }
}

To utilize these new helpers we can parse the input in our Cmdlet to the corresponding enum, and then call the GetUrl method with the instances resource specified in the admin API
Then we'll use the Uri to instantiate a new Authentication class.

Enum.TryParse(Location, out DataCenterLocations tenantLocation);
var serverUrl = UrlFactory.GetUrl("admin.services", tenantLocation, "/api/v1/instances");

Wrap-up

In this part we created a new class project for our PowerShell module and added some scaffolding to it. In the next part we will look into how the authentication works and flesh out the AuthenticationHelper class.

Wednesday, August 9, 2017

Connect to Dynamics 365 using MFA without app tokens, part 2

In part 1 of this series I explained how to use my slightly modified version of the SharePoint-Sites-PnP webauthentication.
The complete code and updates to the powershell module can be found on my github repository

In this post I'm going to dive into getting cookie authentication to work with OrganizationWebProxyClient

First off, our goal for this is to get an IOrganizationService inherited connection that we can use to execute MSDYN365 requests. To do that we have to use one of the connection classes available which allows us to use cookies for authentication.
Unfortunately, the SDK does not contain any constructors which enables us to set cookies programmatically, so we have to look for alternative ways to get around that limitation.
After navigating through the most common connection alternatives I found that the OrganizationWebProxyClient inherits the WebProxyClient, which again inherits the System.ServiceModel.ClientBase class. This is our best way in because the ClientBase class exposes the endpoint used as a public property, and will limit the amount of "hacking" needed to inject our cookies into the request.
So to start with we simply instantiate a new OrganizationWebProxyClient using the organization service endpoint address.

1
var service = new OrganizationWebProxyClient(new Uri("https://myOrganization.crm4.dynamics.com/XRMServices/2011/Organization.svc/web"), false);

I added a break point to debug the connection created and look at the behaviors defined for the underlying endpoint. My hope was to find somewhere to inject my cookies and test the connection, but unfortunately there was nothing I could take advantage of


Yet another minor snafu in the quest for a simple solution. At this point I do what any decent developer should, and that is to google "endpointbehavior cookiecollection". As usual I found a lot of discussions, degrading comments and unrelated answers, but I also found this little brilliant snippet from Markus Wildgruber in a thread on StackOverflow
Adding this into a little helper class allowed me to inject my cookies into the service endpoint without any more issues. A few lines later and I'm sitting with this pretty little thing:


 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
var webAuthCookies = WebAuthentication.GetAuthenticatedCookies("https://myOrganization.crm4.dynamics.com", Models.AuthenticationType.O365);
var service = new OrganizationWebProxyClient(new Uri("https://myOrganization.crm4.dynamics.com/XRMServices/2011/Organization.svc/web"), false);
var cookieContainer = new CookieContainer();
foreach (Cookie cookie in webAuthCookies)
{
    cookieContainer.Add(cookie);
}
var cookieBehavior = new CookieBehavior(cookieContainer);
service.Endpoint.EndpointBehaviors.Add(cookieBehavior);

service.Execute(new WhoAmIRequest());

Unfortunately, my joy was short-lived. Executing this request presented me with a 401 error saying the following:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
System.ServiceModel.Security.MessageSecurityException occurred
  HResult=0x80131501
  Message=The HTTP request is unauthorized with client authentication scheme 'Anonymous'. The authentication header received from the server was 'Bearer redirect_uri=https%3a%2f%2flogin.windows.net%2fcommon%2fwsfed, realm=Microsoft.CRM'.
  Source=mscorlib
  StackTrace:
   at System.Runtime.Remoting.Proxies.RealProxy.HandleReturnMessage(IMessage reqMsg, IMessage retMsg)
   at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
   at Microsoft.Xrm.Sdk.IOrganizationService.Execute(OrganizationRequest request)
   at Microsoft.Xrm.Sdk.WebServiceClient.OrganizationWebProxyClient.<>c__DisplayClassd.<ExecuteCore>b__c()
   at Microsoft.Xrm.Sdk.WebServiceClient.WebProxyClient`1.ExecuteAction[TResult](Func`1 action)
   at Microsoft.Xrm.Sdk.WebServiceClient.OrganizationWebProxyClient.ExecuteCore(OrganizationRequest request)
   at Microsoft.Xrm.Sdk.WebServiceClient.OrganizationWebProxyClient.Execute(OrganizationRequest request)
   at <this line has been omitted on purpose>

Inner Exception 1:
WebException: The remote server returned an error: (401) Unauthorized.

I have to admit, I scratched my head for a long time, trying all kinds of silly things, before I remembered that I was dealing with web requests here, and that meant that the next natural step was to try and use fiddler.
I set an early break point, started up fiddler and hooked it to my process to try and see what was happening. I noticed that when sending the request the cookies seemed to be added to the header correctly, but I also noticed that there was a security header there that I didn't add myself.

This reminded me that when I checked the endpoint behaviors earlier there was one behavior for the client credentials. I fired up the debugger again and dived into the endpoint behaviors, and as I suspected there was a header there with no value since I was doing cookie injection instead of traditional authentication. I went into my cookie behavior and added this little snippet to get rid of the unnecessary header:

1
2
3
4
if (httpRequestMsg.Headers.AllKeys.Contains("Authorization"))
{
    httpRequestMsg.Headers.Remove("Authorization");
}

Firing up the debugger again, I set went to my break point and hooked up fiddler again. This time the Authorization header was missing, and to my pleasant surprise the WhoAmIRequest returned a 200 OK. Going into the results view I could see that I had successfully retrieved the userid, businessunitid and organizationid.

Finally! A working OrganizationWebProxyClient that I could use, and all of this without having to register the application with Azure AD.

Now I can finally go back to writing my Dynamics365-PoSh module without having to worry about authentication until version 9.1 comes out and removes the endpoint I used.

Finally, I would like to give a huge shout-out to Mikael Svenson. I would have spent considerably longer to figure out all of this if he hadn't let me bounce some ideas off him.

Connect to Dynamics 365 using MFA without app tokens, part 1

Lately I've been working on a PowerShell module for MSDYN365 which is meant to simplify administration and mundane tasks like approving mailboxes and adding document locations. As part of this effort I wanted to support MFA authentication as many of the functions I'm creating requires tenant admin, and tenant admins (should) always have MFA enabled for their logins.

Usually when you build an app that authenticates against microsoftonline you need to add an Azure AD application, and use the id and key in your app to enable users to log in. Alternatively you can ask the user for credentials and do the authentication in the background, but this might be difficult in cases where MFA is enabled. Luckily for me, I'm writing PowerShell modules, which means the code is running in the users' context. That means I have access to any credentials or cookies that are added, and I can reuse them without having to register anything in Azure AD.
The reason why I want to avoid AAD registration is because I want to make this easy to use and as portable as possible. Downloading a PS-module and then having to modify it with AAD tenantid, app id and app key is not very user friendly, even if you only have to do it once.

Luckily for me, the guys working on the SharePoint-PnP framework had already done a lot of work on getting authorization working by capturing the cookies after a log on session in a browser, and reusing that for their own purpose. By borrowing their code and making some very minor adjustments to it I got authentication working with redirect for MSDYN365 Online, and I put it into a little helper class named WebAuthentication.
What this code does is that it loads wininit.dll to prevent using any existing authenticated sessions.
In addition, it sets the persistcookie option to false, which means that none of the cookies collected will be persisted.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
[System.Runtime.InteropServices.DllImport("wininet.dll", CharSet = System.Runtime.InteropServices.CharSet.Auto, SetLastError = true)]
public static extern bool InternetSetOption(int hInternet, int dwOption, IntPtr lpBuffer, int dwBufferLength);

private static unsafe void SuppressWininetBehavior()
{
    /* SOURCE: http://msdn.microsoft.com/en-us/library/windows/desktop/aa385328%28v=vs.85%29.aspx
        * INTERNET_OPTION_SUPPRESS_BEHAVIOR (81):
        *      A general purpose option that is used to suppress behaviors on a process-wide basis. 
        *      The lpBuffer parameter of the function must be a pointer to a DWORD containing the specific behavior to suppress. 
        *      This option cannot be queried with InternetQueryOption. 
        *      
        * INTERNET_SUPPRESS_COOKIE_PERSIST (3):
        *      Suppresses the persistence of cookies, even if the server has specified them as persistent.
        *      Version:  Requires Internet Explorer 8.0 or later.
        */

    int option = (int)3/* INTERNET_SUPPRESS_COOKIE_PERSIST*/;
    int* optionPtr = &option;

    bool success = InternetSetOption(0, 81/*INTERNET_OPTION_SUPPRESS_BEHAVIOR*/, new IntPtr(optionPtr), sizeof(int));
    if (!success)
    {
        MessageBox.Show("Something went wrong");
    }
}

To use the code simply call the static GetAuthenticatedCookies method, which takes the crm-server url as an input, and an authentication type. In the Dynamics365-PoSh project I've hard-coded it to use o365, which means it will collect cookies from microsoftonline instead of using claims authentication with ADFS.

1
var webAuthCookies = WebAuthentication.GetAuthenticatedCookies(ServerUrl, Models.AuthenticationType.O365);

What happens next is that a windows forms window will pop up and navigate to the URL given. This will redirect to login.microsoftonline.com, and ask you for your credentials like you are used to. After you've logged in you will get the redirect to the MSDYN365 home page. When this happens "Navigated" event triggers, and then the "ClaimsWebBrowser_Navigated" method will close the form as the authentication is ended.
What we're left with is a cookiecollection which allows us to send authenticated web requests to MSDYN365


In the next post we'll explore how to use this to instantiate an IOrganizationService connection to MSDYN365 using the cookies collected.

Wednesday, August 2, 2017

Form validation failure in App for Outlook for Tasks

Today I encountered a strange error when trying to create a new task from an email in the MSDYN365 app for Outlook. On the form I was required to put in the activity status of the task, which I wasn't allowed to, and it gave me a curious error message just saying "yt"


 


Thanks to the magnificent Scott Durow I found out that this is an error caused by a business rule that comes out of the box with new deployments of MSDYN365 Online. To fix it simply find the business rule on the task entity which is named "enter rule name", and deactivate it.
It comes along with the Project Service Automation solution, so there is no way to delete it for now.

Hopefully we'll see a fix soon.

Wednesday, June 7, 2017

Default view in subgrid resets to system default after edit

I just found a small bug (probably) when you edit the default view from a subgrid in a form. When you close the edit window (whether you made any changes or not), the default view resets to the system default.
To provoke this do the following steps:

  1. Open a form editor for a form with a subgrid (or just add one).
  2. Open the subgrid control, if you already use the system default then change to another one (and save)
  3. Click the edit button
  4. Close the edit view popup
  5. The default view resets to the system default
This isn't a big thing, but if you don't pay attention you're suddenly publishing a form change into production that isn't supposed to be there, and depending on your deployment routines it could be some time before you have the opportunity to fix it.

EDIT: Also realized a much larger issue. When this happens it also resets the "selected views" (if selected). That means that selected views will reset from your selected ones to ONLY the system default view (e.g. My Activities).

Also, now you know that it isn't you who did something wrong.


Tested and verified bug since at least CRM2015, still included in the current edition.

Video of the bug in action:

Thursday, May 18, 2017

Showing and hiding sections on a form + quirks

Showing and hiding sections on a form is not possible to do in a fully supported way without hiding all components of the section.
This can be quite arduous, and in addition someone might have created a business rule that messes that up for you and all of a sudden, it's visible anyway.

To help with that there's a great little function hidden in the client side library that you could use
Xrm.Page.ui.tabs. get(delegate: MatchingDelegate<T>): T[];
This little bit lets you query for a tab using a filter in form of a function. This will let you specify criteria like section content and other properties/attributes.
This means you loop through all tabs on the form and filter by the criteria specified. I’m doing the following (please notice that I’m writing in typescript):
var parentTab = parent.Xrm.Page.ui.tabs.get(t => {
    return t.sections.get("MyUniqueSectionName") !== null
});
var mySection = parentTab[0].sections.get("MyUniqueSectionName");
mySection.setVisible(false);
For the first part, I’m creating a delegate function of t, where t is the current tab being iterated over.
Inside the delegate there’s a short one-liner which returns true if there is a section in the tab with the unique name. This will generate an array of tabs as a result set. I made sure that I used a unique name for my section so I will only get one tab in return.
Next I collect the section from the result set, and finally set the visibility to false.
Putting this into a reusable function with two parameters, I can now utilize it as follows:
setSectionVisibility("MyFirstUniqueSectionName", false);
setSectionVisibility("MySecondUniqueSectionName", true);
Now I can reuse the function and trigger it on field changes or form load to make sure I only show the sections applicable for a given scenario.

Quirks!

I found one quirk with this which was kind of annoying, but luckily the fix isn’t particularly hard. If you have a business rule which shows and hides fields in a section, and that section is hidden at the same time (for example if you trigger on change of a field, and both the business rule and this javascript snippet triggers on the same change), then the following quirk happens.
The section stays visible, but if you write out console.log(mySection.getVisible()) it will say “false”. I assumed there was some race condition going on, because if I hid a field with a business rule at the same time as I hid the section with this javascript, then the section stayed visible but the field got hidden. I did some testing and found out that if you put it inside a setTimeout() then it works like a charm. What I ended up doing was this:
setTimeout(f => {
    setSectionVisibility("MyFirstUniqueSectionName", false);
    setSectionVisibility("MySecondUniqueSectionName", true);
}, 10);

Even though the timeout is only 10 milliseconds that’s still enough to prevent the race condition, and everything works brilliantly.


Another thing I found is that the tab has another, undocumented function: Xrm.Page.ui.tabs. getByFilter(delegate: MatchingDelegate<T>): T[];

This does the same thing as get(…), but has a longer, more explicit function name. Seeing that this is undocumented I would stay away from it as long as both do the same thing (using Daryl Labar’s DefinitelyTyped definitions for Xrm in Typescript will only show the documented function).


Edit:
I have been notified that the xrm definitions of DefinitelyTyped is an effort made by the following:
David Berry
Matt Ngan
Markus Mauch
Daryl LaBar
Tully H

Refreshing web resources and iframes, the supported way

I wrote this great (IMO) little snippet to refresh web resources and iframes (which is basically the same thing) on an MSDYN365 form.
A great thing about the client side library is that it allows you to control form components even when you're operating from inside another frame (be aware that this is also a risk, and something you should be aware of).

I made a web resource which lists out data from several different child-entity records, lets call it RecordLister, and then I have another web resource which allows for creation of different child-entity records, lets call it RecordCreator.

When I create new child records from the RecordCreator I want to refresh the RecordListener resource, but as you might know the "Xrm.Page.data.refresh" function does not reload the web resources, it only reloads the data in the native fields. However, since the web resources are basically iframes it should be possible to reload the content, and luckily for us there is.

First off, I'm doing this in typescript, and I'm using the excellent DefinitelyTyped definitions made by Daryl Labar. You should too!

So here's the code I used to get this to work, and I'll explain it step by step.


var recordLister = parent.Xrm.Page.ui.controls.get("WebResource_recordlister") as Xrm.Page.FramedControl;

What this bit does is just getting the web resource using the Xrm client library, and since I’m using DefinitelyTyped I’m casting it to the specific type I know it is (in this case a FramedControl).

var oldSource = recordLister.getSrc();

Next I’m getting the source url from the control and store it in a local variable, followed by setting the source to about:blank. This way I’m keeping the relative URL that was used before, and I’m changing the source value to make sure the client recognizes the change (setting the value to the same thing doesn’t trigger a reload). I use about:blank just to make sure that should anything crash, hang or otherwise work improperly then at least the frame is blank.

recordLister.SetSrc(oldSource);

Finally I’m setting the source back to its original value, triggering a reload of the contents.

So that’s an easy way to reload your web resource or iframe using supported javascript (typescript) client side functions.

Edit:
I have been notified that the xrm definitions of DefinitelyTyped is an effort made by the following:
David Berry
Matt Ngan
Markus Mauch
Daryl LaBar
Tully H