What is a BizTalk Administrator

This is a question i get all the time. What is a BizTalk Administrator?

A BizTalk administrator has a lot of important roles to take care of. A good BizTalk Administrator should have good knowledge of Microsoft servers, SQL, PowerShell, IIS, .NET, infrastructure, Network and preferably Oracle, fileshares, WCF and other adapters used by your company.

The biggest difference between a BizTalk developer and Administrator is that the administrator has nothing to do with developing integration (basic file integrations can be created by the administrator). The administrator should have good knowledge of BizTalk, how it works, how to run health checks, stress testing, migrating integration and know how to create good integrations. It's so sort of an Architect job, but at the same time not.

The Administrator should have the full responsibility of accepting new integrations, integrations violating the framework should not be accepted, the ability to suspend integrations giving to many errors not defined in the documentation of the integration. Or even integrations giving the environment problems or issues should be suspended.

The administrator should have full knowledge of the health of the environment and how to tune it, there is no environment the same, so a full knowledge of the environment running at the company should be well known for the administrator.

Knowledge of the adapters being used, how they work at how to check for errors on different adapters, a knowledge of the most common errors and what they mean. The administrator should also know how to set up a complete environment from scratch and how to do a disaster recovery if everything goes wrong. Knowledge on how to set up the backup jobs, and the different SQL job required for BizTalk, a good knowledge of the BizTalk SQL is also impotent.

Creating PowerShell commands to enhance the productivity of BizTalk. How to export multiple messages at the same time (BizTalk has these powershell method already made for you, all you need to do is change them for your need).

A BizTalk administrator should be well known of throttling, and how to change edit and debug possible throttling errors.

Performance Counters are many for BizTalk, which one are impotent to monitor at all times. And how do you read them, a BizTalk administrator should know this. How does a DFS fileshare work, what are known bugs and possible scenarios if you create a lot of receive locations and send ports towards one file server?

How to set up a "Microsoft best practice" environment, is this something for you or do you have to do it another way?

How many servers should you be running, what will happen in the next two years, are the integrations able to migrate to a newer BizTalk version.

How does SCOM work, how can it enhance your stability and monitoring. How does the BizTalk pack for SCOM work, and how can you use the benefits from this in your everyday work.

Should you use a third party monitoring service like IPM, Minotaur or BizTalk Monitord (I recommend the last one). Do you need tracking on some if the integrations, what are you tools for this, how do they work?

This is just a little of it all, more will come later. So keep following.

Best wishes!

Useful programs for a BizTalk Administrator

There are a couple of programs that makes the day easier for a BizTalk Administrator.

MessageBox Viewer
This is a product from Microsoft. MessageBox Viewer simply does a quick and vital health check of your environment. Generates a report and a file for Terminiator (another program from Microsofts tech guys).

Its and easy to use GUI and should be run at least once a month to make sure everything is running smoothly.

If errors occurs there's usually an explanation behind the error on how to solve it!

MessageBox Viewer can be downloaded here:
http://blogs.technet.com/b/jpierauc/archive/2007/12/18/msgboxviewer.aspx

Terminator
This is another tool to do simple queries towards the BizTalk database to clean up some of the errors discovered in MessageBox Viewer eq. Orphan messages, tracking and so on.

Terminator for BizTalk can be downloaded here:
http://blogs.msdn.com/b/biztalkcpr/archive/2009/10/01/biztalk-terminator-download-install-info.aspx

BizTalk Monitord
BizTalk Monitord is an open source monitor solution for BizTalk created by me. It gives you the information regarding the hosts, ports and receive locations. Easy to use, SMS warnings and so on. You can add many environments and users.

SMS warnings can be turned on if you use clickatell.

BizTalk Monitord can be downloaded here:
http://sourceforge.net/projects/biztalkmonitord/files/

The benefit of using the MMC

As I've mentioned before, using MMC (Microsoft Management Console) will make it easier to administrate all your environments,

Note: It is only compatible to one version, running BizTalk 2010 and 2009 at the same time will not work

This is what you should have installed.

SQL Server Management Studio
I use this alot to work towards the BizTalk SQL to make my own queries.

Administration Tools Pack
This will give me the required tools for the MMC console.
Can be located here:

Windows XP: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=c16ae515-c8f4-47ef-a1e4-a8dcbacff8e3
Windows 7: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=7d2f6ad7-656b-4313-a005-4e344e43997d
Windows Vista: http://www.microsoft.com/downloads/en/details.aspx?FamilyId=9FF6E897-23CE-4A36-B7FC-D52065DE9960&displaylang=en  

IIS
Installing IIS on the local computer will give the the IIS functionality in MMC

BizTalk Administration Tools
Also include the WCF option during installation.
This is all you need as for now. Remember to not install the entire SQL or BizTalk this will aggregate license costs.

When this is done you need to reboot your computer.

What you need to do now is to hit start choose run and write MMC. A MMC window til show up now save the new window under C: somewhere, in my example i stored it under C:\BizTalkApps\ and named it BizTalkAdministration.mmc.

Create a new shortcut on the desktop. Here's the values for the target:
runas /user:<domain>\<username> "C:\BizTalkApps\BizTalkAdministration.mmc"

For me this would be:
runas /user:ts\admin_tord "C:\BizTalkApps\BizTalkAdministration.mmc"

When you now start the new shortcut created on the desktop a DOS window appears and ask you for a password, type it and the MMC console opens up. As you can se its still blank. Choose File -> Add / Remove snap in. Add all the elements you want. And your done.

How to create a good BizTalk framework - Part 1

What is important, what will be the correct for you and your BizTalk environment?

In order to make a framework you should have a good knowledge of BizTalk. Not just as a developer but also as an Administrator. The difference is huge, and having a good knowledge about both could become very helpful!

Keep the names of your colleagues out of the framework, they might not work there tomorrow!

Sometimes hiring a senior BizTalk Administrator to do this job could be the right for you. You are free to contact me for more information and BizTalk administrator i can recommend for this job.

Having a good framework documented will make it easier when you hire new people and getting them up to date on how you work because, BizTalk is not used the same everywhere. It will be easier to hand over a framework to consultants whom just are in for a little while.

First you need to determine what kind of integrations you have, do you need "100%" uptime, or is it okay if you turn them off for a couple hours, maybe during night, or weekends? You need to take this to consideration when you want to complete a framework for BizTalk.

Do you have a local development office, or do you order them "out of office" ? What demands can you set, and what are your current requirements?

How are you working, are you currently using PowerShell to implement new integrations, or are you doing it manually, is it working good enough?

What kind of environments do you have currently, do you have a test, pre installation and production environment?

Recomended environment:
Development Environment - Not reachable for BizTalk Administrators
This is the environment where the development takes part; this is where only the developers have access.
Pre-installation environment - Not reachable for developers
Will the integration install successfully, or will it fail during installations, this is a clean environment, running this on VMWare or Hyper-V
Test environment - Not reachable for developers
This is where you do a full scale test of the integrations, test messages and "stress testing" should be done here.
Production environment - Not reachable for developers
Little or less needs to be said about this environment. Its production.

My test and production environments are identical, they all have the same assembly files (unless some newer are under testing) the servers are set up the same way, same hardware and same BizTalk installation (can be run under another domain as the production to cut off some costs for licensing, same goes for the Pre-Installation environment.

My BizTalk servers in test and production is a total of 4, 2 single and 2 clustered. The clustered server have the responsibility for the WCF and other adapters requiring for HTTP or IP on receive.

I can control the production environment from my local computer using MMC (I will post more information regarding this).

There are a few topics that will be covered in the framework:
Change Log
Keep the document well sorted, and well documented, keeping a change log is vital to make sure the documentation is up to date, and recent changes can be looked up easy.

Purpose
What is the purpose of this framework, why do you need it, and why should you follow it?

Prerequisites
What do you need in order to make sure this document can and will be followed, except from a BizTalk environment, do you need and other departments to follow this framework?

Responsibilities
Many have the BizTalk administrators divided into doing different tasks; one may have the responsibility of taking care of new integrations, error handling, health checking and so on. Describe all the different tasks that needs to be done (keep names out of the document since people tend to change jobs every now and then)

Describe the tasks in full detail, example of tasks;
- BizTalk Integrations
- Error handling
- Installing new integrations
- health check

New integrations
What should be done when new integrations comes from development, in my example the integration must first be installed in the pre installation environment. If this goes bad the integration is returned back to development for fixing. If it goes well, with no errors it should be installed in test, when installing it in test we should have test messages and correct documentation in hand. Maybe you prefer to have two weeks of testing to make sure you have tried all combination of the new integrations, what happens if the message is empty? What happens if you remove something from the file? What happens if a part of the messages is not received? When should it land under suspended in BizTalk, should you maybe have an error catalog for some type of errors?

Changes to existing integrations
What do you need to take into consideration, as I mentioned earlier the questions is, what if the integration needs to be up every day, except for one day a year, maybe more or maybe even less? (PowerShell can do this for you very easy). What do you require, can the new installation be installed over the old one, do you have versioning on all assembly and file paths?

Handover Procedures
What are your requirements when you get a new integration, do you have a meeting with the developers, going thru the whole integrations or is documentation good enough?

In general, the new integration or changes to existing integrationsDescribe in general your requirements for handing over new or changes to exsisting integrations.

Delivery of the integrations from development to management
Describe in depth what you requirements are for all integrations handed over from development to management.

Documentation requirements
Do you have documentation requirements, where can they be located?

Checklists
When a new integrations comes in, do you have a checklist the integrations must pass before you can install it in production? If not make one, and follow it!

Quality assurance of new integrations
How can you as a BizTalk administrator give assurance to the users of the integration that is fulfills the requirements you have, and they expect?

Approval of new integrations
How do you approve new integrations? Describe in depth.

Implementation of new integrations
How do you implement new integrations, are you using PowerShell, or are you installing the integrations on its own, is it some integrations that needs to be done one way or the other? If you have any of these, describe the type and why it needs to be done this way.

Internal testing of new integrations
How do you test the integrations, what tools are you using, are you manipulating messages to see the affect it has, do you let the developers test?

Integration Environment
Here you need to write about the environments, write some general information, what kind of environments do you have, where are they located, what are the hardware, how do they work? Clustered or not? What are their names and DNS names. What are the environments called? A good Visio drawing could help a lot.

BizTalk Hosts
I assume you have some hosts, write them all down, you should also mention that you are running receive locations with one user, orchestration with one user and send with another, security is important people!
Example:
ReceiveHost64
This is a receive host running 64-bit. This host has the responsibility of all files and is running on all the servers at the same time.
IsolatedHost
This is an isolated hos, this host is only running on server1 and server2, these two servers are clustered. This host only runs WCF receive locations.
OrchestrationsHost
This is by default a 64-bit host at hour environments. These hosts run on all the servers and have the responsibility to do the orchestrations of all integrations.
And so on…

Installation Guide for BizTalk Server
How do you install BizTalk servers at your company, do you have special requirements for hardware, do you have special requirements for where the SSO should be and how the MSSQL should be setup, this is the place to put this into writing.

Roadmap for Integration Environment
This category has 2 sub categories
- Current target image
What are your current target
- Future target image
What are you future goals?

Web services
Web services are getting used more and more, you should have some information on how they are set up at you company.

Setup Web farm
How is it setup, is it running on one server, several, clustered and how!

Name Standard
Remember to always have a good standard for naming integrations, both WS and BizTalk integration should have its own preset way. This makes it easier to work with, and easier to troubleshoot.
You have to sub categories:
- BizTalk
- WebServices

Troubleshooting
How do you troubleshoot, there are a few sub categories here, you can add more if you need them.

- Deviation Handling
How do you do the deviation, send emails, register them in some ITIL or similar procedures?

- What defines a deviation

I usually like to say that anything that is not normal operation is a deviation.

- Errors

What are your procedures during errors, if a message is suspended (maybe its described in the documentation) if the BizTalk environment is giving out errors, what do you do?

- Residual error

I often call integrations that gives me residual errors for “problem integrations” integrations providing a known error, that occurs over and over again is a risk to the environment and should be taken care of as soon as possible.

- Many suspended messages

What do you do if you have to many suspended messages, have you stress tested your environment and know how many you can handle?

- Too many active messages

What do you do if a message is running active for too long, and what if they pile up?

- Orphaned messages

What are your routines on orphaned messages, this is a known issue in BizTalk 2006 – 2009. How do you check for them and how often. When do you remove them?

- Patching (Microsoft security patch and BizTalk patching)

How is patching being done by you, do you take test first? Do you need to test all integrations before implementing the patches to the production, or is the patching going automatically?
- Health Check
How and when do you perform a health check do you have one with Microsoft?


Monitoring
How do you monitor BizTalk, do you have programs, who have access and what are they doing?

Manual Monitoring
What is your manual routines for monitoring?

Automatic monitoring
What is your automatic routine for monitoring (could I recommend my BizTalk Monitord? Can be located on sourceforge.com)

Change and upgrade procedures
How do you change and upgrade your environments, do you do a migration to a new set of servers, do you have any certain vulnerabilities?


Versioning of applications
What are your stands on versioning the applications? Assembly files, and other required files for integrations?

More topics will be given later, for now. Good night.

Retry exhausted Error code 5649

Event error code: 5649

What happens?
Retry exhausted, and receive locations falling down, a BizTalk administrators nightmare!

You might find this error on any of the BizTalk servers with error code: 5649:

Event Type: Error
Event Source: BizTalk Server 2006
Event Category: (1)
Event ID: 5649
Date: Date
Time: Time
User: N/A
Computer: ComputerName
Description:
The receive location "FileReceive" with URL "\\ServerName\FileDrop\FileName.FileNameExtension" is shutting down. Details:"The FILE receive location \\ServerName\FileDrop\FileName.FileNameExtension exhausted the network retry attempts. ".

The solution is as easy as the problem!
First of all, you need to make sure you have checked if all the servers have the right registry settings. You need to make changes to the fileshare and the biztalk server (oh yes, all of the servers). If you have the fileshare clustered you need to do it on all the physical servers.

This is what you do on the file servers, log on to them, and open the registry.
locate the following folder:
Locate the following path in the registry:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters

See if you can find a REG_DWORD with the name MaxMpxCt, if not create one and add a number at least 3 times more then what you have pointing towards the fileshare. (you can add the value to 5000). The defualt value is 50.

If the parameter allready is there, manipulate it into the desired value. After this is done, restart the server, and verify that the value (registry change) is still there.

Now for the BizTalk servers:

Locate the following path in the regisrty:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanworkstation\parameters

See if you can locate a REG_DWORD named MaxCmds, if not there create it and put the value to at least 3 times the locations you have pointing towards the fileshare, (remember that send ports actually do count as well).

If you are unsure you can put 5000 here, if the parameter allready is registered just manipulate the value into 5000 (or desired value). Defualt valute for this paramter is 15.

When you have done this restart the servers and verify that regisry change is still there.

You can read more about the error here:

http://support.microsoft.com/kb/952556

Feel free to let me know if you need any more help regarding this problem.

BizTalk Monitord

Many people have been asking more about my project BizTalk Monitord.

If you need to contact me regarding this, please read the bottom of this post.

BizTalk Monitord is as of the moment opensource, and free to use. There are some known bugs, but be patient it will be solved.

I've gotten some questions that i want to answer.

1. Why choose PHP?
Many reasons, more challenging and easier to keep competitors of my back, even harder to stealing my code and ideas!!

2. Can it run on IIS?
Yep!

3. Do I have to use clickatell for SMS warning?
On the current version, yes. And no i do not make any money on it!

4. Can we get support?
Yep, all you need to do is contact me (see bottom of this post)

5. Can you help me install it?
Yep, see bottom of the post for contact information.

6. Whats comming up next?
Full health check, Error checker, more information about this later.


My contact information:
Tord Glad Nordahl
Phone: 0047 902 40 908
Email: tord.glad.nordahl@tecksolution.com