Wednesday, February 24, 2016

Using and mocking OrganizationServiceProxy (part 1)

How to use the OrganizationServiceProxy with Dynamics CRM, and mocking it

This is a two-part blog series about how to use the OrganizationServiceProxy class with MSCRM. I'll demonstrate how to connect using explicit, hard coded credentials as well as using the service credentials. I'll finish up by giving some tips on mocking the OrganizationServiceProxy to simplify unit testing in your projects.
In part 1 we'll look at utilizing the OrganizationServiceProxy and creating some code which allows us to easily and flexibly integrate with MSCRM.

Prerequisites

To follow the steps described in this post you'll need to have a version of Microsoft Visual Studio as well as the Windows Identity Foundation framework added to your operating system.
Visual Studio is available in a free (community) version found here
Windows Identity Foundation can be activated with the methods described here

Using OrganizationServiceProxy

Set up the project

I'm going ahead and creating a new Web Application project in Visual Studio. I'm not worrying about hosting and which functions I'll need, so I'll just set up a simple MVC project with defaults. I'm also going ahead and creating a unit test project at the same time, which will be used to demonstrate mocking a bit later on.


When the project has been created, open up the nuget package manager and search for Microsoft.CrmSdk.CoreAssemblies. Add this package to both the MVC project and the Test project. You can also add it using the package manager console with the following commands:
Install-Package Microsoft.CrmSdk.CoreAssemblies

Next add a new, empty controller to your MVC project named CrmController. In the index method we're gonna start by defining a new OrganizationServiceProxy with values as described:

public class CrmController : Controller
{
    // GET: Crm
    public ActionResult Index()
    {
        var crmUrl = new Uri(@"https://crmviking.crm4.dynamics.com");
        var clientCredentials = new ClientCredentials();
        authCredentials.ClientCredentials.UserName.UserName = "username@domain.com";
        authCredentials.ClientCredentials.userName.Password = "password";

        var service = new OrganizationServiceProxy(uri: crmUrl, homeRealmUri: null, clientCredentials: authCredentials.ClientCredentials, deviceCredentials: null);
     
        return View();
    }

}

With just a few lines of code you've already got a working service context which can be used to send and retrieve from Dynamics CRM. I'll explain the different inputs to create a new organizationserviceproxy:
uri: This is the URL to your Dynamics CRM instance
homeRealmUri: This is the WS-TRUST URL to your secondary ADFS server, for example if you're federating across domains. I'm not using it in my case but it could be applicable in your case.
clientCredentials: This is the user credentials used to authenticate with CRM.
deviceCredentials: This is for when you generate device credentials for your service

Refactoring service generation

Now, the next logical step (to me) is moving this out into it's own class, so we can reuse for our other methods. What we're doing is generating new a new service context based on predefined values, so we'll refactor the into it's own CrmServiceFactory class. At the same time we'll extract the credentials values and put them into our web.config file (how to store and use your credentials is a discussion better left for another post, but out of two evils, I'd rather specify them in the web.config than hard coded in your class).
Add the following lines to your web.config, inside the "Configuration" namespace.
<appSettings>
  <add key="CrmUserName" value="name@domain.com" />
  <add key="CrmPassword" value="password" />
</appSettings>
<connectionStrings>
  <add name="CrmWebServer" connectionString="https://crmviking.crm4.dynamics.com" />
</connectionStrings>
<configSections>


Refactoring our code into a factory for generating a new OrganizationServiceProxy gives us the following factory-code:

public static OrganizationServiceProxy GetCrmService()
{
    var crmUrl = new Uri(ConfigurationManager.ConnectionStrings["CrmWebServer"].ConnectionString);
    var authCredentials = new AuthenticationCredentials();
    authCredentials.ClientCredentials.UserName.UserName = ConfigurationManager.AppSettings["CrmUserName"];
    authCredentials.ClientCredentials.UserName.Password = ConfigurationManager.AppSettings["CrmPassword"];

    var creds = new AuthenticationCredentials();
    var service = new OrganizationServiceProxy(uri: crmUrl, homeRealmUri: null, clientCredentials: authCredentials.ClientCredentials, deviceCredentials: null);

    return service;
}

Now we can change the implementation in our controller to simply:
var service = CrmServiceFactory.GetCrmService();


Using service credentials

If we want to use service credentials we start by specifying which credentials will be used to run our application. For an MVC application we do that by specifying the user account settings in the IIS Application Pool. More information about setting the service credentials in IIS is described here (technet).
Next we need to change our code implementation as follows:

public static OrganizationServiceProxy GetCrmService()
{
    var crmUrl = new Uri(ConfigurationManager.ConnectionStrings["CrmWebServer"].ConnectionString);
    var authCredentials = new AuthenticationCredentials();
    authCredentials.ClientCredentials.Windows.ClientCredential = CredentialCache.DefaultNetworkCredentials;

    var creds = new AuthenticationCredentials();
    var service = new OrganizationServiceProxy(uri: crmUrl, homeRealmUri: null, clientCredentials: authCredentials.ClientCredentials, deviceCredentials: null);

    return service;
}

As you can see, what we've changed is replacing the explicit declaration of the username and password and converted to using the credentials that our application is running with.
This way we won't be relying on hard coded values, and we don't risk "giving away" our credentials if somebody snatches up your source code.

Using the organizationserviceproxy

First of, technet has a lot of information and examples on how to use the CRM components, and I highly recommend that you spend a fair amount of time reading up on them. There's a lot more to coding against CRM than using classes in .Net. Here's the url to the OrganizationServiceProxy

Implementing a create method

OK. We'll just create a super simple class which will create an account. We'll name it AccountRepository.

public void Create()
{
    var service = CrmServiceFactory.GetCrmService();
    var account = new Entity(entityName: "account");
    account.Attributes["name"] = "Contoso";
    service.Create(account);
}

That was simple, good to go right? Not quite, if I left it at that the Marvelous Hosk would throw harsh words my way. We have some basic principles we should adhere to, mainly Dependency Injection. We'll modify our code to take in the service in the default constructor, and we'll take the name used to create the account as input for the "Create" method.

private readonly OrganizationServiceProxy service;
public AccountRepository(OrganizationServiceProxy service)
{
    this.service = service;
}
public Guid Create(string name)
{
    var account = new Entity(entityName: "account");
    account.Attributes["name"] = name;
    var accountId = service.Create(account);

    return accountId;
}


OK, that's a bit better, we can reuse the class in different projects without having to rewrite any logic, and we can create accounts with different names as well. In addition, we're returning the unique identity (Guid) of your newly created account, which is useful in a number of different scenarios.

Implementing a retrieve method

Implementing a retrieve method is really simple. We'll just add a method to our existing class as follows:

public Entity Retrieve(Guid id)

{

    var account = service.Retrieve("account", id, new ColumnSet(true));

    return account;

}

That's easy and self explanatory, but unfortunately it requires us to know the account id of the account we're retrieving, and I for one do not go around remembering the Guid of my accounts.
So what we'll do is that we'll change this implementation to querying CRM for an account based on the account name, because that's a value we'll remember. The only thing is, when we do a query we'll retrieve a list of entities. Querying for the account name will potentially give us multiple accounts as a result, so I think we'll go ahead and create a new method, named RetrieveByName.

public EntityCollection RetrieveByName(string name)
{
    var query = new QueryExpression("account");
    query.ColumnSet = new ColumnSet(true);
    query.Criteria.AddCondition(new ConditionExpression("name", ConditionOperator.Equal, name));

    var accounts = service.RetrieveMultiple(query);
    return accounts;
}

Now we're retrieving a collection of entities, if we wanted we could also just return a generic list of entities, but I would rather do that elsewhere in my code than implement logic here which makes the method more rigid and less reusable.

Implementing an update method

As you might expect, updating entities aren't much harder. I'll jump straight into implementing an Update method which an entity. It takes an Entity as input, which means we'll be doing the main manipulation in other classes. This might seem redundant in our example, because we're not doing anything that we couldn't do by just calling the OrganizationServiceProxy's Update method. For most deployments that's probably all you need as well, but for some scenarios you might want to do additional, mandatory manipulation every time an update is performed. You might want to whenever it's called, or you might want to implement a date field which is supposed to be updated whenever an update occurs. Additionally, I like to handle all my organization queries inside my repositories.

public void Update(Entity entity)
{
    service.Update(entity);
}

Easy peasy.

Implementing a status update method

Updating the status of a record is a bit special for Dynamics CRM. Instead of simply updating the state and status using the update method you have to send a SetStateRequest.
Here's the code we'll implement.

public void UpdateStatus(Guid id, int state, int status)
{
    var stateRequest = new SetStateRequest();
    stateRequest.EntityMoniker = new EntityReference("account", id);
    stateRequest.State = new OptionSetValue(state);
    stateRequest.Status = new OptionSetValue(status);

    service.Execute(stateRequest);
}

There's no magic in this code either, but as you might notice it is quite generic. We're already taking in the entity id, the state value and the status value. The only parameter missing is the entity logical name and we could reuse it across all entities, and that's exactly what we'll do. A point I want to make is that we'll be passing in four parameters, and to stay in Uncle Bob's good graces we'll create a model to pass as the input instead.

First off, here's our model

public class CrmStatusModel
{
    public Guid Id { get; set; }
    public string EntityName { get; set; }
    public int StateValue { get; set; }
    public int StatusValue { get; set; }
}


Next, it's our new, generic status update class. I went ahead and named it CrmStatusHandler. Like our repository, I'm going to pass in an organizationserviceproxy in the default constructor.

private readonly OrganizationServiceProxy service;
public CrmStatusHandler(OrganizationServiceProxy service)
{
    this.service = service;
}
public void UpdateStatus(CrmStatusModel model)
{
    var stateRequest = new SetStateRequest();
    stateRequest.EntityMoniker = new EntityReference(model.EntityName, model.Id);
    stateRequest.State = new OptionSetValue(model.StateValue);
    stateRequest.Status = new OptionSetValue(model.StatusValue);

    service.Execute(stateRequest);
}


Now we can use this handler to update the status for all our entities, and we've also got a model instead of four separate parameters.

Create additional entity repositories

Now we've seen how to implement a repository for the account entity. I'm gonna go ahead and create another repository for the contact entity. I'll implement the same methods as we did in the account repository, with the same input parameters, except for the query by name.

private OrganizationServiceProxy service;

public ContactRepository(OrganizationServiceProxy service)
{
    this.service = service;
}
public Guid Create(string name)
{
    var contact = new Entity("contact");
    contact.Attributes["name"] = name;
    var contactId = service.Create(contact);

    return contactId;
}

public Entity Retrieve(Guid id)
{
    var contact = service.Retrieve("contact", id, new ColumnSet(true));
    return contact;
}

public void Update(Entity entity)
{
    service.Update(entity);
}


As you can see, it's pretty much the same as the account, only for the contact entity. In addition, I'll create two methods for querying by values instead. I'll create one method for querying by first name, and one method for querying by last name.

public EntityCollection RetrieveByFirstName(string name)
{
    var query = new QueryExpression("contact");
    query.ColumnSet = new ColumnSet(true);
    query.Criteria.AddCondition(new ConditionExpression("firstname"ConditionOperator.Equal, name));

    var contacts = service.RetrieveMultiple(query);
    return contacts;
}

public EntityCollection RetrieveByLastName(string name)
{
    var query = new QueryExpression("contact");
    query.ColumnSet = new ColumnSet(true);
    query.Criteria.AddCondition(new ConditionExpression("lastname"ConditionOperator.Equal, name));

    var contacts = service.RetrieveMultiple(query);
    return contacts;
}

Create an adapter

Lastly, we'll create an adapter to utilize our repositories. I'm going to simulate a situation where we'll always create a contact whenever an account is created, and we'll create a method to deactivate a company when a contact is deactivated. These aren't necessarily methods you'd want to implement in an actual useful environment, but it's a good example of where you'd want to utilize an adapter pattern to combine the usage of several repositories.

public void CreateCustomers(string accountName, string contactName)
{
    var service = CrmServiceFactory.GetCrmService();
    var accountRepository = new AccountRepository(service);
    var contactRepository = new ContactRepository(service);

    var accountId = accountRepository.Create(accountName);
    var contactId = contactRepository.Create(contactName);


    var contactId = contactRepository.Retrieve(contactId);
    contact.Attributes["parentcustomer"] = new EntityReference("account", accountid);
    contactRepository.Update(contact);
}

public void DeactivateCustomers(Guid contactId)
{
    var service = CrmServiceFactory.GetCrmService();
    var accountRepository = new AccountRepository(service);
    var contactRepository = new ContactRepository(service);
    var statusHandler = new CrmStatusHandler(service);

    var contact = contactRepository.Retrieve(contactId);
    var accountReference = (EntityReference)contact.Attributes["parentcustomer"];

    var contactStatus = new CrmStatusModel()
    {
        EntityName = "contact",
        Id = contactId,
        StateValue = 1,
        StatusValue = 2
    };
    var accountStatus = new CrmStatusModel()
    {
        EntityName = "account",
        Id = accountReference.Id,
        StateValue = 1,
        StatusValue = 2
    };

    statusHandler.UpdateStatus(contactStatus);
    statusHandler.UpdateStatus(accountStatus);
}

The first thing you might notice is that these two methods have some redundant code. That already gives you an inclination that there potential for refactoring and improvement. There's some immediate changes we could make, mainly the number of instantiated classes and yet again breaking the dependency injection rules. The first thing we'll do to reduce the redundancy and get better DI is that we'll add the OrganiationServiceProxy as an input for the public constructor for our adapter. Then, in the public constructor we'll set up our repositories as private readonly properties, so they're available to both of our methods inside the adapter. Another thing to note is that the second method also uses the status update handler we created earlier. Creating a new class instance is cheap, especially when we've already got the OrganizationServiceProxy for our adapters, so I'm going to instantiate the status handler in the constructor as well, even though we might not even use it for a particular instance of the adapter class.

private readonly AccountRepository accountRepository;
private readonly ContactRepository contactRepository;
private readonly CrmStatusHandler statusHandler;

public CustomerAdapter(OrganizationServiceProxy service)
{
    accountRepository = new AccountRepository(service);
    contactRepository = new ContactRepository(service);
    statusHandler = new CrmStatusHandler(service);
}


As you can see this hasn't reduced the amount of lines mentionably, but we've got control of the instances at the top of our class declaration, and we can easily change or manipulate them in the future without changing the values inside each method. We'll do some more with our code in the next part, which is Unit Testing our new classes using Moq, so if you've got objections to the changes just made I'd check that out first.

Disposing your objects

Remember that the OrganizationServiceProxy creates new network connections, and you should always call the dispose method when you're done (or instantiate it in a using statement). The network connections aren't part of the CLR, so even in your MVC/web api project where your controllers are instantiated and thrown away in milliseconds the connections will stay open until the idle time out occurs.

Wrap-up

In this part we've looked at how we can utilize the OrganizationServiceProxy to integrate with Microsoft Dynamics CRM. We've created some repositories, a generic status handler and mixed all of our classes into a nice, extensible adapter class. In part 2 we'll look at unit testing these classes, and mocking the OrganizationServiceProxy using Moq. To do that we need to take a look at interfaces, and you'll understand the decisions made in this part even better.


Until then, happy CRM-ing! 

Wednesday, February 10, 2016

CRM 2016 Sales Module Overview

The Dynamics CRM 2016 sales module

Dynamics CRM is split into several different predefined modules. The modules were represented as a vertical navigation area in CRM 2011 and was reworked into a horizontal, partly collapsible navigation bar in CRM 2013. Starting with CRM 2015 SP1, which was only available online, and now in CRM 2016 the horizontal navigation bar now expands to reveal both areas as well as related entities.

Today I'll go through the sales module and what's included out-of-the-box. I won't be doing a thorough walk through of each entity and all it's possibilities, but I'll touch on what I find the most useful features and how they are intended to use.

Dashboards, dashboards, dashboards

I've always been a big fan of dashboards, and like previous versions MSCRM comes with a bunch of them available straight out of the box. The predefined dashboards are based on common use patterns and include several useful views and charts. In fact, many users find that these dashboards fits to their established work pattern and don't even bother to customize them. Though I find that thought alluring, I think most will gain from looking into views and filtering to make it a perfect fit for them. After all, customizing CRM when you get used to it is both easy and fun, because it shows you the results right away and has snap-to styling which alleviates the need for design.

Now I'm an idealist, so I believe that users will find the best way for them to work with a system given time and familiarity with the applications, but MSCRM is huge so it's important to give them the best possible baseline to work from. Having the same baseline also helps when another user has to step in when someone is on sick leave or taking a holiday, as well as designing help and training materials for new employees.
That's why I like dashboards, because it enables you to design a work space for your users that will give them immediate access to the items they work on, whether they're looking into leads, opportunities or sales orders, with the tedious searching and sorting already taken care of. This puts responsibility on the stake holders to invest the time and resources needed to find out how the users of your company works, and analyzing those behaviors and habits to design a system which helps them be more effective, without having to force everyone to do everything the same way.

I recommend that you take some time to check out the dashboards that exist, and maybe write down which parts of the dashboards catches your eye the most. By analyzing which views and charts gives you actual insight and which ones that are more "eye candy-ish" you'll be well on your way to start designing your own. I will be doing a whole blog post, or maybe even a series of blog posts, on designing dashboards and defining use cases.

What's new


What's new is a list of the most recent posts. Posts are wall entities that are connected to activity records, and can be manually or automatically created. Automatic creation happens through autopost rules which can be configured. I will be going through this in my customization posts in the (hopefully) new features.
It can be embedded both into entity forms through the social pane and directly in dashboards, so if this is something which is relevant to your company you might want to consider integrating it.

Account and contact

This is usually the basis for all of your CRM content. In most cases all parts of CRM is either supposed to be connected directly to one of these, or designed to support the use of them. Whether it's sales orders, support cases or SLA's it's all based around a customer base consisting of accounts and contacts.
Many retail customers decide to use only contacts or only accounts, deciding either to connect everything to contacts or customizing the account entity to support retail customer information. I usually recommend persons are created as contacts, and if you're retailing then create a retail account for your contact and add a custom option set to the account which indicates whether it's a company or a person. This is simply because personal details are "flimsy". People change their name, phone number(s), address, email, etc. Using accounts for orders and cases and so on allows you to check if a seemingly new customer might have existing orders, cases or whichever other entities you use. Contacts often get duplicated, and you'll often get new leads based on email addresses they haven't used before or missing middle names etc. Not to mention, people have the same names, so using this technique allows you to have similar names for contacts, which most likely is the field you'll be using in your views/lists, but with unique account names.

I just want to underline that this in no way is a de facto standard, and there are probably many who will disagree with this design, but I'm sticking to "my beliefs" until someone convinces me otherwise. I'd also like to point out that in CRM, both Account and Contacts will be available in the "Customer" lookup, but that's also the reason for my recommendation.

Many retail customers decide to use only contacts or only accounts, deciding either to connect everything to contacts or customizing the account entity to support retail customer information.
I strongly advice to use accounts and contacts as intended, which is exactly as they are described. If you're retailing and customers need to open an account, and they can add several members to that account (for example family members), then you add both contacts and accounts. You could add a option set to the account to specify whether it's a "company" or an account for a customer.
This is particularly true for Dynamics365, which is right around the corner (at the time of writing).

Lead

Here is one of the first steps where implementations starts to deviate from the proposed and even advised usage. A lead is an unqualified opportunity. That's all there is to it. No ifs or buts, it's just a potential, but you haven't qualified the potential at all. It could be someone who's registered on your homepage or a tip from another customer, but it's nothing more than an unqualified opportunity. You can create a new contact and/or account from a lead, and then you can either qualify or disqualify it for an opportunity. There could be a question about what qualifies a lead, you might not want to add a person who has subscribed to your company blog as a lead, but if someone contacts your company and says "I want to buy product X" then that's already past a lead, that's already at the opportunity stage or even order stage in the case of many retail customers.

You might wonder why that's important, and that's simply because Microsoft is designing a lot of functionality around the in-built entities, so if you start using leads, opportunities or anything else for things they weren't designed for then you might miss out on cool new features when the next update comes around.

Opportunity

The opportunity entity is used for working on a proposal for a customer. It could be a qualified lead or it could be a raw opportunity created from the overview. What's important to know about the opportunity is that it should be qualified, meaning that it could only be won or lost. That's the only status options available, and the only possible outcomes if you use the opportunity like it's designed. For example, "I have an opportunity to sell this company my products or services, either they'll buy from me or they won't. They might come back later or request multiple quotes, but it's either a sale or not." If you think about it this way then an opportunity can only be won or lost, and losing an opportunity does not necessarily have anything to do with the performance of the seller.
During the opportunity phase you present the customer with one or more quotes, and if the customer decides to buy then you convert the quote(s) to a sales order, and at that time you close the opportunity (when the last order is placed). When an order is placed you have won the opportunity, whether it's actually paid or not.
Please note that MSCRM is just that, a customer relations management system, it is not meant as an accounting system. There are other, great Dynamics products which can solve your accounting needs, as well as other providers that can be integrated with CRM.

When working on an opportunity it is often recommended to use a business process flow (BPF), a feature introduced in CRM 2013 (and largely improved since then). I will be doing a future blog post on best practices and possibilities for using BPFs in your CRM system. I will also be going more in-depth in regards to using opportunity, with focus on developing, proposing and identifying factors like competitors, stake holders, etc.

Competitor

Competitor is pretty self-explanatory. It's an entity used to create competitor records that can be used in opportunities to identify who you're competing with.

Quote

Quotes are used to present a customer with a solution. It is tightly integrated with both opportunities and order lines. You add a number of order lines, costs and description inside a quote, present it to a customer (for example through automated emails), and if the customer accepts that quote you can convert it directly into a sales order. When a quote is created you can add information like address, order lines, discount, etc. When the quote is finished and ready to be presented you click the button to activate it, and at this point you're not allowed to rework it. This is a way to make sure that a quote does not get worked on from the moment it is presented to the customer and until you either retract it or get a request for revision from the customer. At this point you can simply click the revise button to edit the quote details and/or order lines, and present it again.
Do note that closing a quote also allows you to create a new, revised quote.
If the customer accepts the quote you can create a sales order directly from the toolbar, which will include all the details that originally was in the quote.
Here's one of the points about what I mentioned earlier about what won means in Dynamics CRM. When you choose to convert the quote to a sales order you can set the opportunity to Won at the same time. This is because as far as CRM is concerned, a sales order means a sale has occurred, which also underlines how quotes and sales orders are supposed to be used. Do not create a sales order before the customer has confirmed that the quote will be purchased.

The quote has a lot of different features, and I highly recommend using it. I will be doing several posts on customizing the sales area to fit customer needs, where I'll go more in-depth into the quote and sales order entity.

Sales order (or just order) and invoice

You can think of the sales order as an extension of the quote, and invoice as an extension of a sales order. They both contain the same information as the quote, and has the same connections as the quote.
Remember not to fulfill an order before all the invoices have been paid. Even partial fulfillment will close the order, and at that time you can't go back to open (without cheating, but we're not gonna go into that in this post).
That's another reason for closing an opportunity when an order is created. Because the order should be active until all invoices are paid there's no need to keep the opportunity open, because it has been won and you're finished with it. For many companies there will even be different teams working on opportunities and sales orders, so just close that opportunity as soon as possible after the order is created and start working on your next prospect!

Also take note that you can, and should, lock the pricing. This is because as soon as an order is placed you've probably agreed to the price, and at that time you shouldn't allow the price to be changed through recalculation when the unit price changes or the currency conversion rate changes.

Product

The product entity has a lot of capabilities, and needs to be thoroughly thought through and planned before you start implementing any. There's a lot of possibilities, among them bundling products, product properties, price lists and on it goes. That's why I'm saving it for another post to keep the length of this one manageable. 

Sales literature

The sales literature is an entity for creating a wide range of different sales literature. It has many predefined types, like spreadsheets, general sales literature, price sheets and manuals. Sales literature should have attachments that could be used by your sales people to read up on, or product manuals that you can attach to emails and provide your customers with.
It's also connected with competitors and products, because sales literature will probably be directly relevant to either a single or a series of products, and you might have competitors who provide the same or similar product.
The usage of sales literature varies a lot from company to company, but it is a great way to gather information about your products and offer a way to give your sales people the newest presentations and general news for your products. Combine this with mobile clients and they'll be able to answer customer requests on the go, or even directly in meetings if the need arises.

You could also create sales literature for a campaign you're running, for example price sheets, or products you're only offering for a limited time, and specify an expiration date. You can also set an expiration date to products that for whichever reason aren't available anymore, in case you still want the literature available.
If you have literature for products that your own employees either have created or have the main responsibility for then you can set up an employee contact. This gives your users an easy way to find the correct contact person when they have a question about the manual or suggestions for revision.


The wrap-up

You made it to the end! The sales module is quite large, and we only covered the entities primarily used for sales in this post. There are several others, like goals, quick campaigns and alerts that are used across several of the modules in Dynamics CRM.
Using the sales module the way Microsoft has designed it allows you to work effectively with a work surface you're already familiar with. I think it's a great product, and hopefully this post has made you feel the same way if you didn't already.

Stay tuned for more posts on Dynamics CRM, in February I'll be doing several posts on configurations in CRM On-Premises and Online, and I'll finish strong with some posts on migrating from the former to the latter.

Until next time, happy CRM-ing!

Saturday, February 6, 2016

Create CRM 2016 Online instance

Creating a CRM Online instance (with Office365)

This blog post will go through the details of creating a new CRM Online trial subscriction.
It is one of the steps on my "February CRM roll" where I'm setting up CRM 2016 on-premises, going through a lot of configuration and migrating over to CRM Online, data, config plugins, the works!
This part is all about setting up Office365, and it's a lot simpler than you think, in large parts thanks to the guidance the office365 portal gives you.

Sign up for a new trial

First of all, head over to the trail website (Microsoft)
Click the Get Started button under "Self-Start Trial" to start the registration.

Step #1. They'll want to know a little about you, like the country you're in, name, phone, company, etc. I presume you'll know how to fill out these fields yourself.

Step #2. Create a user name and instance ID. The username will be the tenant administrator, and will be your login name. The tenant name will be appended to .onmicrosoft.com to generate a complete username. In my case it's: crmviking@crmviking.onmicrosoft.com

Step #3. Confirm your details by letting them text you a confirmation message, or receive an awkward, automated phone call. If you're interested about the other online products you can check the boxes for email, phone, etc.

Creating your account details will only take a minute or two, so don't run off yet!

Step #4. Log in with your new account details, and you'll be presented with a form to confirm some details like language, country and currency. REMEMBER! The currency cannot be changed later, so stick to the one you'll want as your base (additional currencies can be added, but the base will always be the same). I recommend that you choose the country you'll be working from, so you will get a tenant with the lowest possible latency.
It'll spend a few minutes creation your account, but when it's done you'll be presented with a link to your CRM environment.

Congratulations! You have your own CRM Online environment with a bit of sample data to play around with!

Add more services

Ok, so we have our new CRM Online environment up and running, and even though that's alluring we'll do some more housework first. If you head over to the Office365 portal you can get an overview of your services. If you're asked for a username and password it's the new one you just created:
username@instance.onmicrosoft.com

As you can see, you have 4 services available. CRM, Identity Service, Office365 portal and Social Engagement. These are all the default services that comes with CRM Online. That's fine, if CRM is all you need, but I'm want more services available so I can make use of all the cool functions CRM has to offer, like synchronization of email, contacts and tasks, as well as sharepoint and onedrive for business integration. And you won't be wrong to assume that I'll be doing a future post on powerbi integration with MSCRM Online either.

So, head over to the "Purchase Services" over on the left hand side. Don't worry, we'll just be adding trial services, so there's no credit card requirements or anything like that. So, we're currently running a CRM Online 30 day trail, but there are loads of additions to choose from. I recommend that you spend some time reading about the different products, like Dynamics Marketing and Parature. It might not be something you're interested in at this time, but they full of really cool features that you'll want to keep in mind as your business' digital needs evolve and expend.
As I mentioned earlier, I would like to add some integration features, so I've scrolled down to the Office365 E5 plan. The E3 plan is, by far, the more popular choice, but I want to show off how incredibly cool it is to integrate with Power BI.
So, just hover your mouse over the E5 plan, and click on the "Start free trial" option.
You'll be presented with a confirmation order and instrutions to assign licenses to your users. We'll be doing that a bit later, and I'll also go through adding new members and setting up some basic settings. When you return to the overview you'll see additional services available.

In my case you can see a warning about the Power BI service being degraded. I've expanded the warning which shows that the Microsoft techies have already located the error and are applying a fix. No matter which provider you choose as your cloud platform provider you will experiences issues from time to time, especially if you want to be on the fast-track for new, exciting features. I think this is a brilliant example on how Microsoft gives you the information up-front, with additional information in regards to what they are doing to remediate the issue. In this case, the issue was that some users could experience issues when importing excel files into Power BI, so it was not even a major issue but you're still getting the information so you know that it isn't your system that's the problem. This isn't something revolutionary, but being honest and proactive is a lot better than many other providers who just stamp their products with "Beta" or something like that and tell you that you're using their products at your own risk.

But enough with the preaching, head over to the "Users" section in the left hand navigation, and open up active users. Click on your user account and you can see the different options you have available. At this time, I'll be adding additional features to my user account, but I advice you not to do that in your production account. I would rather have that account as a dedicated administrator, without any licenses applied. Give the account an insanely difficult password, and activate multi factor authentication with the phone number to one of the stake holders in your company. That way, the tenant admin will be secure, and is not used for daily, mundane tasks that increases the risk of credentials being stolen or man-in-the-middle attacks that could give an attacker the opportunity to delete all your stuff, or even worse act on your behalf.
But as I said, in my case I'll just be adding licenses to the default user, because this is just a demo environment. Click on the "Edit" link on the right hand side under "Assigned licenses"


Check the box for E5 license license, and click the save button to continue. At this point a mailbox will be assigned to the user, as well as permissions to download the Office 365 suit for desktop and access to the other cool features (power bi, sharepoint online, etc).

Setting up your Office 365 portal

OK, so we've added CRM Online and the rest of the Office365 suite, as well as adding licenses to your user. What I recommend that you do is navigate through the Office365 Portal and get familiar with the options available and what you can do. I could probably do a a dozen blogposts just to cover all the possibilities, but that would be a bit pointless.
The next step, at least in this guide, is adding a domain and some users. I'm going to go through the "Setup" from the left hand navigation to complete these steps. It's an easy way to get your environment up and going. As you might see from the coming screenshots I've always taken the liberty of styling/theming my Office365 portal as well.
Click the "Start Setup" button to start the Offie365 setup. Just a quick tip early on, if you need to migrate mail content to your O365 portal then you should plan that before you go ahead. It's important that your users are aware of the transition and able to verify that they're able to access all the services they need after the migration is done. I don't have an environment to transfer from, so I'll just go ahead and select that option and push on through.
Now! The next step is to add domains to your deployment. I'm going to go ahead and add the crmviking.com domain, which will allow me to send and receive emails, as well as getting a more user friendly username. You are also able to buy a domain from this step, Microsoft will direct you to godaddy.com to complete the registration of a new domain if you choose to do so. In my case, I already own the crmviking.com and crmviking.no domain, so I'll just go ahead with the wizard.

You'll be provided with a request to add a txt record to verify your domain. It gives some straight forward instructions on how to do that, but you could also contact your domain provider and ask them to help you perform the required configuration changes. When the changes have taken effect, come back and verify your settings. if you've done everything correct you'll be presented with a success screen, and you can continue to updating user IDs.

I'm just going ahead and marking my only account for update, effectively switching usernames from "crmviking@crmviking.onmicrosoft.com" to "crmviking@crmviking.com". If you have several users you can select all for update, or just pick out a selection. If you don't want to update any users at this time you can go ahead and skip, else proceed to the success screen.
If you changed your own account, you'll have to sign out and sign in with the new account name to continue with step 3.

Now, if you had to go through this step, it might have required you to close the browser and try again. In that case, you'll be directed to your user start page in Office365. To get back to the administration site simply scroll down and click on the "Administration" tile.
Head back to the setup area and start the setup wizard again. The next step is adding additional users. I will be adding a dummy account and proceeding, but it's also possible to import a CSV file at this point to mass create users.

Next you will be presented with a screen with the new users created and a temporary password. You can use this to send your users their new account names and passwords, or you can reset the password for them later. Personally I don't like storing passwords, even temporary ones, so if I'm not distributing them straight away I'll just reset them later if need be. Proceed to the next and final step, adding DNS records to use your domain in Office365.
At this step you'll be prompted if you want to transfer DNS management to Office365. There are, of course, both pros and cons to doing this. If you change then you'll be changing the DNS servers you want to use to handle your domains DNS records. This makes everything available from the Office365 portal, which can be handy. On the other hand, you might be more comfortable using your existing DNS servers, in which case you can continue using them and just add the DNS records yourself. I'll be opting in to use my existing servers, because this is a demo environment and I like the domain registration I have at my current registrar.

On the next page, select which services you want to use in Office365 (which requires DNS records). I'll go ahead and select mail and Skype for Business, I do not want to manage my mobile for this demo, and head over to the next page. The next page lists all the DNS records which needs to be created.
Like instructed earlier, you can either update these yourself or ask your domain registrar for help.

That's quite a few, so do what you've got to do to get those registered, and get back to the setup and click next to verify the records. Now, there might be a trick to get the SRV-records to work. If you can't specify the service and protocol at your registrar's DNS settings, you have to specify _sip._tls.yourdomain.com as the service name. John White did a great blog post on this.
And now you're finished! Rate the service/experience, and rock on through back to the office portal.


Wrap-up

And there you have it, CRM Online with an Office365 Enterprise plan created and fully functional. I'll be doing more blogposts on this throughout February to show how you can administer users and licenses, as well as permissions and sharing content with groups. I will also be doing a migration from CRM 2016 On-Premises to CRM Online, with data migration, configuration, plugins. Stay tuned for more CRM goodness!

Thursday, February 4, 2016

Handling CRM 2016 organizations (and a tuning tip)

Handling organizations in MSCRM 2016

This post is about handling organizations in Dynamics CRM 2016 using the Deployment Manager tool and SQL Server Management studio. Just a heads up, this will be along one, but there's lots of pictures too.

What is an organization?

An organization is... well, an organization! You can think of it as an instance in your CRM deployment. Multiple organizations can exist in the same deployment of CRM services, but they are completely separated from each other. The benefit of having multiple organizations is mainly for enterprise size companies, who have large organizations with almost completely different needs in regard to customization and work methologies. When this is the case you can create multiple organizations with it's own set of customizations and web parts.
In addition, if you have multiple developer teams working on different things in your CRM environment then each team can get their own organization to deploy their changes in, and it doesn't require multiple servers with CRM and SQL Server, dozens of AD groups, etc.

An organization is actually just a SQL Server database, as we will see later in this post, and that is part of the reason why you are required to have unique organization names.

Organization overview and deployment administrators

To get an overview of your organizations you can simply open up the Deployment Manager from a CRM Server with the "Deployment Server" roles installed. One caveat is that you'll have to be added as a deployment administrator first, and you need login permissions on the SQL Server where the organization and configuration database is stored, and permissions to the MSCRM_CONFIG database. If you try to open the deployment manager without these permissions first then you'll get the following error message
To add new deployment administrators, log on with the user account used to install MSCRM, and open up the deployment manager tool. Navigate to "Deployment Administrators" and click on the "Add Deployment Administrator" link on the right hand side. Type in the name of the user you want to add, and click OK.


Now, to find your organizations simply navigate to "Organizations" on the left hand menu, and you'll get a list of all the connected organizations in your environment, both active and disabled (but not deleted, more on that in a little while). The overview lets you see the name (this is the unique name), display name, status, version and update availability of all your organizations.


You can also right click the organization to open up the properties for it

Adding organizations

Next let's step through the process of adding a new organization. From the organization overview simply hit the "New organization" link on the right hand side. This will give you the "New Organization Wizard", which collects the basic information needed to create a new organization.
Now, here are a few steps to complete, starting with display name and unique database name. Remember _MSCRM is appended to the unique database name, so it will look like this in SQL Server: myorganization_MSCRM. The display name is the name visible in the navigation bar beneath your name on the right hand side (depending on your screen resolution).

Next choose the base currency. Remember! None of the settings below "Unique Database Name" can be changed after the organization has been created. 
If you click the browse button and find your country from the list it will automatically fill in the currency code, name, symbol and precision.
Now, here's a pro tip: If you install additional language packs on the server then you're able to deploy multiple organization with different base languages. There are a lot of people who think that once the base language is chosen you have to uninstall to change it, but you really just have to deploy a new organization (which suddenly becomes a problem if you've already added tons of data to the old one, but lets hope you spot the mistake early on).
OK, last one out, the SQL Collation. Make sure you choose the same collation as the default one in your SQL Server. If you don't know which collation it has, ask your DBA. But just to be nice, here's how you do that yourself using tsql:
SELECT CONVERT (varchar, SERVERPROPERTY('collation'));

Next, select the SQL Server you want to store that databases on (it automatically fills in the same server as the MSCRM_CONFIG database is stored on), and the reporting URL. The slightly negative thing about the reporting URL is that it won't check which URLs the other environments use, it will just pick the SQL Server name and add HTTP:// to the front and /ReportServer at the back. So if you want to be sure you're using the correct SSRS server then you can copy that from the details about one of your other organizations (see the part about organization overview).
Please note that adding a new organization and importing existing organizations requires the SRS Data Connector to be installed beforehand. See my previous blog post for more information on installing this component.

Now, rock on through to the summary screen, and hopefully it will be one warning accompanied/succeeded by two green flags. The warning is for data encryption which will be activated, and that you should backup you encryption key.

Just a heads up, it is not unusual for the creation to take a long time during the "Microsoft.Crm.Tools.Admin.ImportDefaultDataAction" and similar screens. I've seen these take well over 30 minutes before so just get yourself a decent cup of coffee and come back later (or stare at it intensively, that's always fun)

When the creation is complete the new organization will be visible in the Organizations overview in the Deployment Manager



Deleting an organization

In this section we'll go through how to delete an organization. I'll show you some related topics along the way like editing the organization settings and the overview in SQL Server.

First off, to delete an organization you need to start with disabling it. Simply right click the organization from the overview in the Deployment Manager, and click disable. After you've done this, right click it again to delete it from the deployment. Please be aware that this does NOT delete the database, it simply removes it from the "organizations" table in the MSCRM_CONFIG, the database is still present and online in SQL Server. One thing you might notice when you've disabled the organization is that a new option is available; edit organization. This allows you to specify new values for an existing organization.


Importing an organization

This chapter explains how to import an organization. This typically happens when you want to clone your production environment into test or migrate from one server environment to another.
Start in the "Organizations" overview in Deployment Manager, and click the "Import organization" link on the right hand side.
This will bring up the "Import Organization Wizard", which automatically lists the organizations available for import on the SQL Server specified. Organizations that already exists in the deployment will not be listed.


Next you'll be able to edit both the display name and the unique database name. Editing the unique organization name does not actually edit the database name, it only edits the unique name stored in the database tables, which is appended to Internet Facing Deployment URLs.

Next specify the SSRS server URL you'll be using for this organization, and proceed to the next screen. Now, the installation will ask you for user mappings. This is for mapping the users in the organization to AD user accounts, and you can either choose automatic mappings or manual mappings. Automatic mapping is best when you're importing into the same active directory domain as it previously was. Manual mapping allows you to to specify all users manually, or create an import schema which you can edit in Excel.


I'll stick to automatic, because there's only one user in the organization.
To proceed with the import you have to map the current logged in user to a system administrator in the organization. If you try to continue without mapping this user you will be presented with the following error message

When this is done you'll be ready to import the organization, and you'll be presented with the familiar CRM process bar. Please note that if you've imported an older database, for example CRM 2015 or a previous update rollup, the database schema will be updated during import


And you're done! Organization imported and everything is (hopefully) nice and dandy. If you go into SQL Server you should be able to see that the databases are present, and the name of your organization has not been edited

Bonus round

Parallelism in SQL Server

A good tip I can't give often enough is the Max Degree of Parallelism (MAXDOP) setting in SQL Server. Some years ago in the "Best practices" documentation for CRM 2011 it was adviced to set MAXDOP to 1, meaning only one thread per SQL statement. This can cause horrible performance problems in CRM, because almost all queries rely on joins and filtering based on those joins, and that requires a lot of work if you can't execute it in parallel. Subsequently, this recommendation was removed, but the practice has continued with many consultants and IT Pros (this could also be because MAXDOP 1 is an actual best practice for Microsoft SharePoint).
So my tip: set MAXDOP to 0, and set threshold for parallelism to 4 or 1/4 of the number of processor cores on your SQL Server (whichever is higher). This is a generic recommendation, so do keep in mind that YMMV.

Developer resources in CRM

One of my favorite new things in the new CRM navigation is the improvements to the Developer Resources, available from Settings -> Customization

This new page gives you a lot of great information, starting with useful links for devlopers, the new WEP API available per instance, the organization id and the unique name, as well as the new discovery web api (and the old SOAP api, but SOAP is like, totally so 2009).



That's it for today, I hope you found this post useful.
Tomorrow I'll do a post on how to create an Office365 tenant and activate the CRM feature.
With the Deploying Microsoft Dynamics CRM Online exam it is increasingly important to know your way around the Office365 instance.
Until then, happy CRM-ing!

Wednesday, February 3, 2016

Backup and restore strategies for CRM 2016

Backup and restore strategies for Dynamics CRM 2016

This post is all about backup and restore. I'll be going into some strategies for backing up and restoring your MSCRM-environment. A common misconception is that backing up the database is all you need, and I'll explain why that isn't the case.

Prerequisites

I'm assuming that you know your server names and have a basic understanding of how to back up files and SQL Server databases. I also assume that you have a way to back up encryption keys and passwords (for example KeePass (official site) ).
I also assume that the user account you're using is part of the "PrivUserGroup" AD group.

Backup routines

Backup your SQL Server Reporting Services encryption key

This isn't CRM! Well no, it isn't strictly CRM, but if you want to be able to restore your magnificent, custom tailored, beautiful SSRS reports then you might want to have the encryption key used by SSRS to connect to it's database. You only have to back this up whenever it changes, which typically is whenever you migrate to a new environment or set up a new SSRS-server, so there's no need to keep doing this on a regular schedule. You should do it atleast once, though.
Log on to your SSRS server and open up the "SQL Server 2014 Reporting Service Configuration Manager" (assuming you're running SQL 2014). Select the report server instance you want to configure, which is the one you specified for CRM during installation (please note that the SSRS instance does not necessarily have the same name as the SQL Server instance CRM uses).

Navigate to "Encryption Keys" on the left hand side, and click the "Backup" button. You'll get a new window asking you where to store the key and to choose a password for it. Now, the key is a file you'll have to save somewhere safe. Now, I'm not going to promote practically free and safe storage in Microsoft Azure, but I would definitely put it in a secure storage area in Azure. The password specified for the file should be stored in your password management system, and if you happen to use MICROSOFT AZURE then you'll have a nice URL you can add to the password entry.</plug>

Backup the CRM encryption key

Now for some good old myth busting... kind of. When CRM 2013 first came around the option to encrypt certain tables in the database was introduced. To be able to use server-side-synchronization and specify passwords for mailboxes required you to enable encryption first, and to enable encryption you would have to connect over HTTPS (well, unless you changed those settings with the orgdborgsettings tool). In CRM 2015, encryption is turned on by default, with a long, automatically generated key, and in CRM 2016 it's no different.
Because of this there are some CRM administrators out there who don't know that CRM2015 and newer will have encryption turned on by default, so if you migrated to a new SQL Server then you'd just import the database and everything would be "fine and dandy". That's not the case anymore, you have to backup the encryption key to be able to do that.
So how do you get the key when you aren't using HTTPS in your CRM environment? If you open up CRM and navigate to Settings -> Data Management and open up data encryption you will get this error:

So I'm going to guide you through how to get past this limitation without having to restarting anything.
What you want to do add an additional binding to your CRM website, an HTTPS binding with a self-signed certificate. Head into Internet Information Services (IIS) and highlight the server, then open up Server Certificates from the home screen.
Open this feature, and on the right hand side click the "Create Self-Signed Certificate..." link, and you'll get a new "Create Self-Signed Certificate" wizard. Fill in the name of this certificate and click OK (both personal and web hosting will work).

Now navigate to the Microsoft Dynamics CRM site in IIS from the left hand side, under "Sites". Click on the "Bindings..." link on the right hand side to get the site bindings configuration window. In this window, you will probably just have one binding beforehand, so click the "Add" button to add a new binding. Choose https as the type, the default port will be fine, and select the newly created self-signed certificate for use.
Click OK, then close, and from the right hand side just click the "Browse *:443 (https)" link to open up CRM with SSL encryption enabled. You will be presented with a warning screen which tells you that the certificate isn't trusted for this site, but that's OK because you just created this certificate yourself, and it's just for this purpose you're using it. Click on the red shield with the text "Continue to this website (not recommended)".

From the navigation bar, find Setting -> Data Management. Click on the "Data Encryption" option to open up the Data Encryption settings. And voila! You have the option to show and change the encryption key. The existing key will probably have alot of foreign characters (depending on your native language), and it's fine to keep it that way, or you can change to a new one. I often just generate a new Guid from powershell and add some additional special characters inside it, but as I said the default encryption key is pretty great considering that it's already all special characters. For this installation I'm going to change it into something that is NOT SAFE!

Now, you might get an error saying you're not part of the PrivUserGroup, if you did get this message then that means you'll have to add your user account to the PrivUserGroup corresponding to your organization, or find someone who can grant you that group membership.

Beware, though, if you have multiple CRM deployments you will have several PrivUserGroup AD groups. To find the one you need to be added to, open up powershell and type in the following two commands:
add-pssnapin Microsoft.Crm.Powershell
Get-CrmOrganization | fl Id

This will list your organization(s) and give you the ID which succeeds "PrivUserGroup" in the group name.


Now, put that encryption key into your password management system (btw: Have you considered using Azure Key Vault for storing stuff like this? Feel free to contact me if you have any questions regarding Azure Key Vault and storing encrypted data).
Finally, go back into IIS and remove the binding that we added earlier. Just follow the same procedure as provided above, except remove instead of add. If you don't want to keep the self-signed certificate (which you shouldn't), then just go to the "manage server certificates" window as described earlier and remove the certificate from the list.

Back up CRM data

This is the easy part, this is simply a SQL Server backup as long as you're following the Microsoft best practices (not changing any system files, not putting any plugins into the GAC/filesystem, etc). Just do your normal SQL Server backups and keep them available for restoration. I would also recommend that you do a new backup of the Master database, because if you environment fail and you restore the databases to a new system then the Master will include information about user permissions and also be allowed to read from the encrypted tables.
If you don't know how to do SQL Server backups then you probably shouldn't do that on your own, and if you want to learn I heartily suggest you check out the work of Ola Hallengren, the king of maintenance jobs on SQL Server.

Restore strategies

Restoring CRM isn't that big of a deal, actually, so I'm going to give you a very quick, 101 on how to restore a failed CRM environment. You will find information on how to do these steps in my previous blog posts and in this one. I recommend that you design your strategy based on your environment, maybe create a shiny step-by-step guide by actually performing the steps in a QA environment.
Remember, having a backup/restore strategy is not the same as knowing how to perform the actions needed when your system fails. Make sure that you've been through the ropes atleast once, and repeat them when necessary. If you're going to be doing alot of development in your environment then I can recommend that you try it out whenever you're cloning production back to development/test/QA. That way you're sure to remember what to do if disaster strikes in production.

  1. Restore your most recent backups of the CRM Databases
  2. Restore the user/group permissions on the SQL Server instance (if necessary). I won't go into the details because you should be aware of what you're doing when you're messing around in SQL Server, and this post would get too lengthy if I included DBA training in it.
  3. Restore the ReportingServices database
  4. Open up reporting services configuration manager (as described earlier in this post). Choose to attach to an existing database, and import the encryption key you've backed up and stored the password for.
  5. Install Microsoft CRM 2016 on an application server, in the server installation choose to connect to an existing instance instead of creating a new deployment. Point to the database server where the MSCRM_CONFIG-database is located
  6. Import the existing organization databases into CRM through the Deployment Manager (only if the databases have been moved to a new server)
  7. Add the https-bindings to your CRM site in IIS as described in the backup steps, open up CRM and navigate to Settings -> Data Management -> Encryption. Enter the encryption key you've backed up in your password management system or similar.
  8. Remove the https-bindings. Make sure that everything works as expected and shabang, your done!

So that's it for today, it's quite the lengthy post (again), so you're entitled to some cake and an extra good cup of coffee.
In tomorrows post I will demonstrate how to add manage organizations in the deployment manager tool and how to get access to them. I will also include some great tips for performance on the SQL Server side (nothing too fancy, mind), so I'm considering that a level200 topic.