Blog is moved!

The blog has been moved to http://www.biztalkadmin.com

How to do a good health check part 2

In my first post you could read about the health check around network and updates provided by Microsoft and other manufactures, IBM, HP Dell and so on.

This part is more related to how to health check BizTalk itself. How to set it up correctly.

The Host separation policy

BizTalk solutions typically consist of executable artifacts - receive locations, pipelines, orchestrations, send ports, and so on. BizTalk hosts provides a way to logically group these artifacts and allow you to instance each host on one or more BizTalk Servers. A Host instance represents a physical run time windows service or process running on a BizTalk. This gives you a very flexible solution to host BizTalk application components, there are many factors which should be taken into account when distributing your applications across hosts.

-Performance

• When you create a new host a set of dedicated databases tables is provisioned in the BizTalk databases. In the BizTalkMsgBoxDB they act as the work queues for that host. This can help to gain low latency situations by creating multiple queues for different workloads, thus presenting messages with low latency requirements getting delayed behind large batches of none "time-critical messages".


• Each host instance has its own set of system resources such as memory threads and handles in the .NET thread pool. Instead of hosting all executable artifacts in a single host i can be beneficial to spread the load across multiple hosts.


• BizTalk throttling is implemented on the host level. You may experience that you need to set a specific throttling parameter in order to control individual artifacts. If you need to do this I would recommend you to separate artifacts across different hosts so that you don't apply unnecessary throttling to all artifacts.

- Functionality

• Its recommended to separate reviving, orchestration, sending and tracking functionality into separate hosts. This will help you to manage and scale each type of processing independently. IF you want to stop all reviving or all orchestration processing. Or adding additional servers to the BizTalk group to handle dedicated orchestration processing only.


• You should also have hosts for application that only can run on one server, POP3, FTP, MSMQ receive. These hosts should be grouped and clustered in the BizTalk console· All others can remain in hosts which supports multiple running host instances.


• Separation application processing across hosts ultimately leads to a multiple runtime processes hosting you applications. As BizTalk can host custom written components (all custom components you have), host separation can be used to provide process isolation and protection. If you experience instability in any custom components, you should isolate them into their own BizTalk host. This should be done to prevent the instability to affect other applications.

- Security

• Each BizTalk host is assigned to a windows group which is used to give the host permission to specific tables in the BizTalk database. This is important when dealing with highly sensitive data.


• Certificates can be utilized in BizTalk for message signing and encryption. This settings is assigned in the host level and therefore you may need to separate hosts in order to represent different identities.


• All host instances are assigned wit ha service account to run under. Access to external systems is often a requirement for this account in order to process outbound messages through the adapters.

Set up a dedicated tracking host
Any hosts that hosts tracking is responsible for moving the DTA and BAM tracking data from the MessageBox database to the BizTalk Tracking (DTA) and BAM Primary Import databases. The moment of tracking data has an impact on the performances of other BizTalk artifacts running on the same host that is hosting tracking. Therefor you should have a dedicated host that does nothing but tracking.

To use a dedicated host for tracking also allows you to stop other hosts without interfering with the BizTalk server tracking. The movement of tracking data out of the MessageBox database is critical for healthy systems. IF the BizTalk host is responsible for moving the tracking data in the BizTalk group is stopped, the Tracking data decode service will not run. Impacts of this is:

• HAT tracking data will not be moved from the MessageBox to the BizTalk Tracking database. Same goes for BAM tracking data, this will not be moved to BAM Primary Import database.

• Because data is not moved, it cannot be deleted from the BizTalkMsgBoxDb.


• When the tracking data decode service is stopped, tracking interceptors will still write tracking data to the MessageBox database. IF the data is not moved, this will cause the MessageBox database to become enormous. which will impact performance over time.


• It is also important for security to have a dedicated tracking host, since the tracking host has access to all the messages across the message box database.


• Turn of tracking on all other hosts, they will still write tracking but they wont have read access.

Isolated Adapter per Application Pool
An application pool which hosts an isolated adapter represents the physical implementation of an isolated host instance. This means that the application pool (w3wp.exe process) becomes an instance of BizTalk because the isolated adapter loads the BizTalk runtime components into the process to perform message processing. BizTalk only supports a single isolated adapter per application pool.

Remove unnecessary BizTalk items, components and settings
It is important to keep a clean environment. Remove old receive locations, send ports and applications that are no longer in use. Artifacts which only were used during testing and development should be removed. The output from the BizTalk Documenter tool can be used for this.

The BizTalk configuration file
BizTalk runtime process is like other .NET applications and have configuration files that can be used to control certain application behaviors. They control runtime validation, orchestration dehydration and provide custom application settings for custom .NET components hosted in BizTalk

BizTalk Registry Key Validation
BizTalk stores a variety of configuration data in the windows registry. This data can have an impact on the behavior of the BizTalk runtime processes and associated services. There is one specific area area which is recommended to tune for the default registry values and these control the .NET thread pool used by BizTalk. You can find the values here:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\BTSSvc$hostname\CLR Hosting

I will add more information in part 3 to 5. There is a lot to go trough and as said earlier, I don't want to spoil it all by giving away everything immediately (plus there is a lot to read).

Check part 1

How to do a good BizTalk Healthcheck part 1

As many of my friends know, my biggest interest is tuning and maintaining a BizTalk environment.

I made this list to make it easier for people to do a good health check. I would advise people to do the run at least once every 6th month. Calling Microsoft "in" to do the job is also recommended if you don't feel safe to do it yourself.. There are also many MVP's to help you out. You are also free to contact me. And i need to mention that i have been working with BizTalk 2006 R2 mostly so this guide is for BizTalk 2k6 and 2k6 R2, but can also be used for newer versions.

If you see something i dont have here, please feel free to leave me a comment.

Patching:
- What are the routines to install windows critical updates.
Good routines are critical.

-You should also run the Microsoft Baseline Security Analyzer (MBSA) on the servers Both SQL and Biztalk server. This is to identify and inconsistencies between the deployed updates and the current list of recommended updates

- Update COM+ and MSDTC.
Both COM+ and MSDTC are used by BizTalk, patches and hotfixes for these services often provide hotfixing to improve tuning and stability.

General Network
- DTCPing of all the servers
There should be no packets lost during this ping.

- Transfer 100mb data
if you have a 1gb network the response should be 5 seconds or faster (average between 3-5 seconds). Do to all the servers. If you are running a 100mb network up to 20 seconds is within the accaptable transfer rate (average is around 8 - 10 second)

- Pathping
Do a pathping from the BizTalk server to all the SQL servers ("pathping

- TCP/IP porth Echaustion
Troubleshoot to see if you are using less then 3000 ephemeral ports. TO do the counting write the following command on the BizTalk Servers "netstat -ano -p tcp" Count the number of unique Local Address TCP ports open above 1024 for each IP adresse. Using TCPView tool makes this is a lot easier.

- DBNETLIB Exceptions
Avoid this, DBnetLib (Database Network Library). The most common error when this occurs is when one of the BizTalk MessageBox becomes extremely busy. Attempts to communicate with the busy MessageBox database results in a timeout. Look for 5410 errors in the EventLog. (example of error message:
Event Type: Warning
Event Source: BizTalk Server 2006 R2
Event Category: BizTalk Server 2006 R2
Event ID: 5410
Computer:
Description:
An error occurred that requires the BizTalk service to terminate. The most common causes are the following: 1) An unexpected out of memory error. OR 2) An inability to connect or a loss of connectivity to one of the BizTalk databases. The service will shutdown and auto-restart in 1 minute. If the problematic database remains unavailable, this cycle will repeat.

-NetBIOS over TCP/IP
This is only a check for the BizTalk servers that have a direct connection to the internet such as those residing in a perimeter network hosting HTTP and SOAP adapters If the BizTalk servers reside in a vulnerable network location then check to ensure any internet/public facing network adapters have NetBIOS over TCP/IP disabled

Server general

-Time sync
Its vital that the time is synchronized between the BizTalk server, and SQL server. Check to see if the time is within the valid synchronization by typing the following command in command prompt "w32tm /stripchart /computer " to resynchronized the clocks type the following in the command prompt window "w32tm /resync /computer "

- BIOS Version
BIOS should be updated, this is because the releases from the manufactures can provide better satbilitet, and network related performance.

- Update the Certificate RevocationListUpdates
If it takes a long time to start up a BizTalk server it may be because you don't have access from the BizTalk server to reach the microsoft.com domain. You might get some startup issues because the .NET framework will try to download the Certificate Revocation LIST (CRL) from microsoft.com. You can update your servers manually by following these two links: http://crl.microsoft.com/pki/crl/products/CodeSignPCA.crl and
http://crl.microsoft.com/pki/crl/products/CodeSignPCA2.crl

I will come to more related to the BizTalk Server Configuration later. I dont want to spoilt it all by giving you everything at once.

Check part 2

Usefull SQL queries part 2

I've gotten a lot of request on more queries. There is another one I use often. The one to determine when during the day you have the most activity.




SELECT datepart(HH, [dtInsertionTimeStamp]) as date,
count(Convert(char(10), [dtInsertionTimeStamp], 108)) as ant FROM [BizTalkDTADb].[dbo].[dta_MessageInOutEvents] WITH (NOLOCK) WHERE [dtInsertionTimeStamp] BETWEEN convert(datetime, '01-01-2011 00:00:00', 120) AND convert(datetime, '07-08-2011 23:59:59', 120) GROUP BY datepart(hh, [dtInsertionTimeStamp])ORDER BY [date]

all you need to do is change the date after the WHERE clause.

Good luck

Useful BizTalk SQL queries

Hello, today I want to share all the SQL queries i use a lot when monitoring BizTalk. As I've mentioned before i use MessageBoxViewer alot, but sometimes i only need some small information, or just a simple query towards the Tracking.

Marklog:
Marklog is a table in the tracking database store a string every time BizTalk backups the tracking database, this is a known bug in BizTalk 2006 - 2009. And it will most probably never be fixed, make sure the marklog table isn't to big, it might give you some performance issues towards the tracking database.

SELECT [MarkName] FROM [BizTalkDTADb].[dbo].[MarkLog] WITH (NOLOCK)

See count of all messages and port name for a specified date. (as long as you have tracking on all receive and send ports.

SELECT CONVERT(char(10), [BizTalkDTADb].[dbo].[dta_MessageInOutEvents].[dtInsertionTimeStamp], 101) AS InDate, COUNT([BizTalkDTADb].[dbo].[dta_MessageInOutEvents].[nPortId]) AS numb, dta_PortName.strPortName AS name FROM [BizTalkDTADb].[dbo].[dta_MessageInOutEvents] WITH (NOLOCK) INNER JOIN [BizTalkDTADb].[dbo].[dta_PortName] WITH (NOLOCK) ON [BizTalkDTADb].[dbo].[dta_MessageInOutEvents].[nPortId] = [BizTalkDTADb].[dbo].[dta_PortName].[nPortId] WHERE [BizTalkDTADb].[dbo].[dta_MessageInOutEvents].[dtInsertionTimeStamp] BETWEEN CONVERT(datetime, '05-10-2011 00:00:00', 120) AND CONVERT(datetime, '05-17-2011 23:59:59', 120) GROUP BY CONVERT(char(10), [dta_MessageInOutEvents].[dtInsertionTimeStamp], 101), [dta_PortName].[strPortName] ORDER BY InDate, numb DESC

The last is just a simple count of all messages registered in the MessageInOutEvents.

SELECT Convert(char(10), [dtInsertionTimeStamp], 101) as date, count(Convert(char(10), [dtInsertionTimeStamp], 101)) as ant FROM [BizTalkDTADb].[dbo].[dta_MessageInOutEvents] WITH (NOLOCK) WHERE [dtInsertionTimeStamp] BETWEEN convert(datetime, '05-10-2011 00:00:00', 120) AND convert(datetime, '05-17-2011 23:59:59', 120) GROUP BY Convert(char(10), [dtInsertionTimeStamp], 101) ORDER BY [date]

Feel free to use these queries as you wish. (remember to change the date in the query).

Make sure the dates are within the time you have tracking stored, and the query execution time will vary depending on how many massages you have during a day and for how long you want to gather the information.

What is a BizTalk Administrator

This is a question i get all the time. What is a BizTalk Administrator?

A BizTalk administrator has a lot of important roles to take care of. A good BizTalk Administrator should have good knowledge of Microsoft servers, SQL, PowerShell, IIS, .NET, infrastructure, Network and preferably Oracle, fileshares, WCF and other adapters used by your company.

The biggest difference between a BizTalk developer and Administrator is that the administrator has nothing to do with developing integration (basic file integrations can be created by the administrator). The administrator should have good knowledge of BizTalk, how it works, how to run health checks, stress testing, migrating integration and know how to create good integrations. It's so sort of an Architect job, but at the same time not.

The Administrator should have the full responsibility of accepting new integrations, integrations violating the framework should not be accepted, the ability to suspend integrations giving to many errors not defined in the documentation of the integration. Or even integrations giving the environment problems or issues should be suspended.

The administrator should have full knowledge of the health of the environment and how to tune it, there is no environment the same, so a full knowledge of the environment running at the company should be well known for the administrator.

Knowledge of the adapters being used, how they work at how to check for errors on different adapters, a knowledge of the most common errors and what they mean. The administrator should also know how to set up a complete environment from scratch and how to do a disaster recovery if everything goes wrong. Knowledge on how to set up the backup jobs, and the different SQL job required for BizTalk, a good knowledge of the BizTalk SQL is also impotent.

Creating PowerShell commands to enhance the productivity of BizTalk. How to export multiple messages at the same time (BizTalk has these powershell method already made for you, all you need to do is change them for your need).

A BizTalk administrator should be well known of throttling, and how to change edit and debug possible throttling errors.

Performance Counters are many for BizTalk, which one are impotent to monitor at all times. And how do you read them, a BizTalk administrator should know this. How does a DFS fileshare work, what are known bugs and possible scenarios if you create a lot of receive locations and send ports towards one file server?

How to set up a "Microsoft best practice" environment, is this something for you or do you have to do it another way?

How many servers should you be running, what will happen in the next two years, are the integrations able to migrate to a newer BizTalk version.

How does SCOM work, how can it enhance your stability and monitoring. How does the BizTalk pack for SCOM work, and how can you use the benefits from this in your everyday work.

Should you use a third party monitoring service like IPM, Minotaur or BizTalk Monitord (I recommend the last one). Do you need tracking on some if the integrations, what are you tools for this, how do they work?

This is just a little of it all, more will come later. So keep following.

Best wishes!

Useful programs for a BizTalk Administrator

There are a couple of programs that makes the day easier for a BizTalk Administrator.

MessageBox Viewer
This is a product from Microsoft. MessageBox Viewer simply does a quick and vital health check of your environment. Generates a report and a file for Terminiator (another program from Microsofts tech guys).

Its and easy to use GUI and should be run at least once a month to make sure everything is running smoothly.

If errors occurs there's usually an explanation behind the error on how to solve it!

MessageBox Viewer can be downloaded here:
http://blogs.technet.com/b/jpierauc/archive/2007/12/18/msgboxviewer.aspx

Terminator
This is another tool to do simple queries towards the BizTalk database to clean up some of the errors discovered in MessageBox Viewer eq. Orphan messages, tracking and so on.

Terminator for BizTalk can be downloaded here:
http://blogs.msdn.com/b/biztalkcpr/archive/2009/10/01/biztalk-terminator-download-install-info.aspx

BizTalk Monitord
BizTalk Monitord is an open source monitor solution for BizTalk created by me. It gives you the information regarding the hosts, ports and receive locations. Easy to use, SMS warnings and so on. You can add many environments and users.

SMS warnings can be turned on if you use clickatell.

BizTalk Monitord can be downloaded here:
http://sourceforge.net/projects/biztalkmonitord/files/

The benefit of using the MMC

As I've mentioned before, using MMC (Microsoft Management Console) will make it easier to administrate all your environments,

Note: It is only compatible to one version, running BizTalk 2010 and 2009 at the same time will not work

This is what you should have installed.

SQL Server Management Studio
I use this alot to work towards the BizTalk SQL to make my own queries.

Administration Tools Pack
This will give me the required tools for the MMC console.
Can be located here:

Windows XP: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=c16ae515-c8f4-47ef-a1e4-a8dcbacff8e3
Windows 7: http://www.microsoft.com/downloads/en/details.aspx?FamilyID=7d2f6ad7-656b-4313-a005-4e344e43997d
Windows Vista: http://www.microsoft.com/downloads/en/details.aspx?FamilyId=9FF6E897-23CE-4A36-B7FC-D52065DE9960&displaylang=en  

IIS
Installing IIS on the local computer will give the the IIS functionality in MMC

BizTalk Administration Tools
Also include the WCF option during installation.
This is all you need as for now. Remember to not install the entire SQL or BizTalk this will aggregate license costs.

When this is done you need to reboot your computer.

What you need to do now is to hit start choose run and write MMC. A MMC window til show up now save the new window under C: somewhere, in my example i stored it under C:\BizTalkApps\ and named it BizTalkAdministration.mmc.

Create a new shortcut on the desktop. Here's the values for the target:
runas /user:<domain>\<username> "C:\BizTalkApps\BizTalkAdministration.mmc"

For me this would be:
runas /user:ts\admin_tord "C:\BizTalkApps\BizTalkAdministration.mmc"

When you now start the new shortcut created on the desktop a DOS window appears and ask you for a password, type it and the MMC console opens up. As you can se its still blank. Choose File -> Add / Remove snap in. Add all the elements you want. And your done.

How to create a good BizTalk framework - Part 1

What is important, what will be the correct for you and your BizTalk environment?

In order to make a framework you should have a good knowledge of BizTalk. Not just as a developer but also as an Administrator. The difference is huge, and having a good knowledge about both could become very helpful!

Keep the names of your colleagues out of the framework, they might not work there tomorrow!

Sometimes hiring a senior BizTalk Administrator to do this job could be the right for you. You are free to contact me for more information and BizTalk administrator i can recommend for this job.

Having a good framework documented will make it easier when you hire new people and getting them up to date on how you work because, BizTalk is not used the same everywhere. It will be easier to hand over a framework to consultants whom just are in for a little while.

First you need to determine what kind of integrations you have, do you need "100%" uptime, or is it okay if you turn them off for a couple hours, maybe during night, or weekends? You need to take this to consideration when you want to complete a framework for BizTalk.

Do you have a local development office, or do you order them "out of office" ? What demands can you set, and what are your current requirements?

How are you working, are you currently using PowerShell to implement new integrations, or are you doing it manually, is it working good enough?

What kind of environments do you have currently, do you have a test, pre installation and production environment?

Recomended environment:
Development Environment - Not reachable for BizTalk Administrators
This is the environment where the development takes part; this is where only the developers have access.
Pre-installation environment - Not reachable for developers
Will the integration install successfully, or will it fail during installations, this is a clean environment, running this on VMWare or Hyper-V
Test environment - Not reachable for developers
This is where you do a full scale test of the integrations, test messages and "stress testing" should be done here.
Production environment - Not reachable for developers
Little or less needs to be said about this environment. Its production.

My test and production environments are identical, they all have the same assembly files (unless some newer are under testing) the servers are set up the same way, same hardware and same BizTalk installation (can be run under another domain as the production to cut off some costs for licensing, same goes for the Pre-Installation environment.

My BizTalk servers in test and production is a total of 4, 2 single and 2 clustered. The clustered server have the responsibility for the WCF and other adapters requiring for HTTP or IP on receive.

I can control the production environment from my local computer using MMC (I will post more information regarding this).

There are a few topics that will be covered in the framework:
Change Log
Keep the document well sorted, and well documented, keeping a change log is vital to make sure the documentation is up to date, and recent changes can be looked up easy.

Purpose
What is the purpose of this framework, why do you need it, and why should you follow it?

Prerequisites
What do you need in order to make sure this document can and will be followed, except from a BizTalk environment, do you need and other departments to follow this framework?

Responsibilities
Many have the BizTalk administrators divided into doing different tasks; one may have the responsibility of taking care of new integrations, error handling, health checking and so on. Describe all the different tasks that needs to be done (keep names out of the document since people tend to change jobs every now and then)

Describe the tasks in full detail, example of tasks;
- BizTalk Integrations
- Error handling
- Installing new integrations
- health check

New integrations
What should be done when new integrations comes from development, in my example the integration must first be installed in the pre installation environment. If this goes bad the integration is returned back to development for fixing. If it goes well, with no errors it should be installed in test, when installing it in test we should have test messages and correct documentation in hand. Maybe you prefer to have two weeks of testing to make sure you have tried all combination of the new integrations, what happens if the message is empty? What happens if you remove something from the file? What happens if a part of the messages is not received? When should it land under suspended in BizTalk, should you maybe have an error catalog for some type of errors?

Changes to existing integrations
What do you need to take into consideration, as I mentioned earlier the questions is, what if the integration needs to be up every day, except for one day a year, maybe more or maybe even less? (PowerShell can do this for you very easy). What do you require, can the new installation be installed over the old one, do you have versioning on all assembly and file paths?

Handover Procedures
What are your requirements when you get a new integration, do you have a meeting with the developers, going thru the whole integrations or is documentation good enough?

In general, the new integration or changes to existing integrationsDescribe in general your requirements for handing over new or changes to exsisting integrations.

Delivery of the integrations from development to management
Describe in depth what you requirements are for all integrations handed over from development to management.

Documentation requirements
Do you have documentation requirements, where can they be located?

Checklists
When a new integrations comes in, do you have a checklist the integrations must pass before you can install it in production? If not make one, and follow it!

Quality assurance of new integrations
How can you as a BizTalk administrator give assurance to the users of the integration that is fulfills the requirements you have, and they expect?

Approval of new integrations
How do you approve new integrations? Describe in depth.

Implementation of new integrations
How do you implement new integrations, are you using PowerShell, or are you installing the integrations on its own, is it some integrations that needs to be done one way or the other? If you have any of these, describe the type and why it needs to be done this way.

Internal testing of new integrations
How do you test the integrations, what tools are you using, are you manipulating messages to see the affect it has, do you let the developers test?

Integration Environment
Here you need to write about the environments, write some general information, what kind of environments do you have, where are they located, what are the hardware, how do they work? Clustered or not? What are their names and DNS names. What are the environments called? A good Visio drawing could help a lot.

BizTalk Hosts
I assume you have some hosts, write them all down, you should also mention that you are running receive locations with one user, orchestration with one user and send with another, security is important people!
Example:
ReceiveHost64
This is a receive host running 64-bit. This host has the responsibility of all files and is running on all the servers at the same time.
IsolatedHost
This is an isolated hos, this host is only running on server1 and server2, these two servers are clustered. This host only runs WCF receive locations.
OrchestrationsHost
This is by default a 64-bit host at hour environments. These hosts run on all the servers and have the responsibility to do the orchestrations of all integrations.
And so on…

Installation Guide for BizTalk Server
How do you install BizTalk servers at your company, do you have special requirements for hardware, do you have special requirements for where the SSO should be and how the MSSQL should be setup, this is the place to put this into writing.

Roadmap for Integration Environment
This category has 2 sub categories
- Current target image
What are your current target
- Future target image
What are you future goals?

Web services
Web services are getting used more and more, you should have some information on how they are set up at you company.

Setup Web farm
How is it setup, is it running on one server, several, clustered and how!

Name Standard
Remember to always have a good standard for naming integrations, both WS and BizTalk integration should have its own preset way. This makes it easier to work with, and easier to troubleshoot.
You have to sub categories:
- BizTalk
- WebServices

Troubleshooting
How do you troubleshoot, there are a few sub categories here, you can add more if you need them.

- Deviation Handling
How do you do the deviation, send emails, register them in some ITIL or similar procedures?

- What defines a deviation

I usually like to say that anything that is not normal operation is a deviation.

- Errors

What are your procedures during errors, if a message is suspended (maybe its described in the documentation) if the BizTalk environment is giving out errors, what do you do?

- Residual error

I often call integrations that gives me residual errors for “problem integrations” integrations providing a known error, that occurs over and over again is a risk to the environment and should be taken care of as soon as possible.

- Many suspended messages

What do you do if you have to many suspended messages, have you stress tested your environment and know how many you can handle?

- Too many active messages

What do you do if a message is running active for too long, and what if they pile up?

- Orphaned messages

What are your routines on orphaned messages, this is a known issue in BizTalk 2006 – 2009. How do you check for them and how often. When do you remove them?

- Patching (Microsoft security patch and BizTalk patching)

How is patching being done by you, do you take test first? Do you need to test all integrations before implementing the patches to the production, or is the patching going automatically?
- Health Check
How and when do you perform a health check do you have one with Microsoft?


Monitoring
How do you monitor BizTalk, do you have programs, who have access and what are they doing?

Manual Monitoring
What is your manual routines for monitoring?

Automatic monitoring
What is your automatic routine for monitoring (could I recommend my BizTalk Monitord? Can be located on sourceforge.com)

Change and upgrade procedures
How do you change and upgrade your environments, do you do a migration to a new set of servers, do you have any certain vulnerabilities?


Versioning of applications
What are your stands on versioning the applications? Assembly files, and other required files for integrations?

More topics will be given later, for now. Good night.

Retry exhausted Error code 5649

Event error code: 5649

What happens?
Retry exhausted, and receive locations falling down, a BizTalk administrators nightmare!

You might find this error on any of the BizTalk servers with error code: 5649:

Event Type: Error
Event Source: BizTalk Server 2006
Event Category: (1)
Event ID: 5649
Date: Date
Time: Time
User: N/A
Computer: ComputerName
Description:
The receive location "FileReceive" with URL "\\ServerName\FileDrop\FileName.FileNameExtension" is shutting down. Details:"The FILE receive location \\ServerName\FileDrop\FileName.FileNameExtension exhausted the network retry attempts. ".

The solution is as easy as the problem!
First of all, you need to make sure you have checked if all the servers have the right registry settings. You need to make changes to the fileshare and the biztalk server (oh yes, all of the servers). If you have the fileshare clustered you need to do it on all the physical servers.

This is what you do on the file servers, log on to them, and open the registry.
locate the following folder:
Locate the following path in the registry:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanserver\parameters

See if you can find a REG_DWORD with the name MaxMpxCt, if not create one and add a number at least 3 times more then what you have pointing towards the fileshare. (you can add the value to 5000). The defualt value is 50.

If the parameter allready is there, manipulate it into the desired value. After this is done, restart the server, and verify that the value (registry change) is still there.

Now for the BizTalk servers:

Locate the following path in the regisrty:

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\lanmanworkstation\parameters

See if you can locate a REG_DWORD named MaxCmds, if not there create it and put the value to at least 3 times the locations you have pointing towards the fileshare, (remember that send ports actually do count as well).

If you are unsure you can put 5000 here, if the parameter allready is registered just manipulate the value into 5000 (or desired value). Defualt valute for this paramter is 15.

When you have done this restart the servers and verify that regisry change is still there.

You can read more about the error here:

http://support.microsoft.com/kb/952556

Feel free to let me know if you need any more help regarding this problem.