Cross Roads

Upcoming SharePoint 2010 Certifications

2010-03-14T01:36:00.002+05:30

SharePoint 2010 Exams coming soon,

For the IT Pros: SP2010 Admin Focus

70-667 TS: Microsoft SharePoint 2010, Configuring
Microsoft Official Curriculum: Will cover configuration of SharePoint 2010 including deployment, upgrade, management, and operation on a server farm.

70-668 PRO: SharePoint 2010, Administrator
Microsoft Official Curriculum: Will cover advanced SharePoint 2010 topics including capacity planning, topology designing, and performance tuning.

For developers: SP 2010 Application Development Focus

70-573 TS: Microsoft SharePoint 2010, Application Development

Microsoft Official Curriculum: Five-day instructor-led course designed for developers with six months or more of .NET development experience. Course covers what you need to know to be an effective member of a SharePoint development team using Visual Studio 2010.

70-576 PRO: Designing and Developing Microsoft SharePoint 2010 Applications
Microsoft Official Curriculum: Five-day instructor-led training course designed for development team leads who have already passed the Developing on SharePoint 2010 technical specialist exam. The course covers choosing technologies for and scoping a SharePoint project, best practices for SharePoint development, configuring a SharePoint development environment, advanced use of SharePoint developer features, and debugging of code in a SharePoint project.

https://partner.microsoft.com/40043180

Finally, SharePoint 2010 running on my Windows 7 Laptop... :-)

2010-03-07T05:18:00.004+05:30

Finally, I got Microsoft SharePoint Server 2010 running on my Window 7 laptop, was suffered a lot with incomplete and inclear steps suggested by Microsoft Guide "Setting Up the Development Environment for SharePoint Server".

I'll suggest to go through comments on the page first, also http://www.mcpblog.net/Lists/Posts/Post.aspx?List=8f2497f6-e77b-4e70-8400-e020b292165c&ID=61 helps to pass through step 8 on configuration wizard.

Update (2010/03/07): download http://connect.microsoft.com/VisualStudio/Downloads/DownloadDetails.aspx?DownloadID=23806

Best of Luck.

SharePoint 2010

2010-02-28T01:30:00.002+05:30

WCF SQL Adapter - BizTalk Server 2009

2010-02-13T20:46:00.000+05:30

WCF SQL Adapter, seems confusing, well terms seems confusing but the adapter is not, The BizTalk Adapter Pack 2.0 now contains fve system and data adapters including

SAP, Siebel, Oracle databases, Oracle applications, and SQL Server.

Till now BizTalk adapter were built using defined BizTalk adapter framework and which based on standard interface and API, now reaching next stage professionals were encouraged by Microsoft to use WCF LOB Adepter SDK. This is new framework which offers evolution of BizTalk conventional armoury, and based of WCF framework.

All adapter in Pack 2.0 built upon WCF LOB meta-data rich components that are surfaced to use WCF bindings, almost similar which you expect from WCF binding. Excellent set of configuration options and flexibility offered by new adapter pack and saving thousands of bucks for connecting premium enterprise application system, e.g. SAP, JDE, other ERP/CRMs etc.

This pack also include SQL Server Adapter, which replace the legacy version of BizTalk as well offer much more flexibility and features. Also it offers significant benefit the previous rage adapter and introduce a range of new capabilities that overdue in realm of database interaction.

Well, I'll try to get some time and elaborate this new excellent chunk of Server 2009.

Setup and Customization of an Axapta Enterprise Portal.

2009-08-16T22:36:00.000+05:30

Yesterday, I tried to collect some of EX-Project experiences and skill in Microsoft Dynamics, I have worked few months ago to setup and Enterprise portal and then to customization of the same portal. At that time it was really fresh and new experience for me particularly with limited help on 4.o version. It was self learning experience and that's why I thought to drive the same experience but with latest version of SharePoint (MOSS 2007). I hope it will be a good start or reading for any new comer in such domain.

Steps need to perform as pre-requisites for setting up Enterprise Portal.
1. Have a look to below prerequisites before you start setup process.

1.1 http://msdn2.microsoft.com/en-us/library/aa493471.aspx, visit the link to see setup guide of MS Dynamics Ax 4.0

1.2 Verify the components installed, and make sure Enterprise portal and supportive components are available to use.

1.3 Make sure other supportive server implementation have all services packs and updates in place if applicable.

1.4 For using reporting services with EP you must to ensure reporting services are well configured and ad-hoc reports also have proper configuration.

1.5 Also it is advice able to uninstall Internet Explorer Enhance Security Configuration.

2. Create Business Connector Proxy account in DC, make sure you create an account with flags like password not expire and to keep more security measures for proxy account you can also flag no interactive rights. You must give following groups right in order to make sure the account works for Business Connector.
IIS_WPG
WSS_WPG
Also it needs any other custom groups if you have plan to access such information falling under those groups inside EP.

3. SharePoint portal or WSS site should have .NET 2.o configuration in IIS.

4. Proxy account which we created for Business connector should have login access of SQL Server as well as it should have dbcreator server role.

5. Configure SharePoint site as integrated windows authentication, you can change the configuration by routing to Application Management -> Application Security -> Authentication Providers. It will list all security configuration, by default it will display 'default' label, you can select active configuration and change to Integrated Windows Authentication with NTLM (EP Portal don't support Kerberos delegation method).

6. Select the application pool in IIS on which EP portal is going to run, check the properties and then Identity tab, change credential to proxy account which we created for business connector. Always rebounce the IIS after any change in contexts by using iisreset /noforce.

7. Open the web.config file of SharePoint file and make sure trust level is WSS_Medium if not then change it to WSS_Medium, you can have custom level of security configuration on need and requirement. It is also advisable to give executionTimeout configuration for 6000 ms for httpRuntime calls.

8. Add below webparts into safecontrol so SharePoint site can use them without any obstructed. IF you are using any custom webparts please make sure they are also listed in safecontrol tags. Recycle the IIS Process after web.config update by using iisreset /noforce command.

Steps for deploy Enterprise Portal.
1. Drive to Enterprise portal configuration wizard (Administration > Setup > Internet > Enterprise Portal > Configuration Wizard), click on wizard and then click on Next button.

2. Configure user groups and map them to Enterprise portal groups. Make sure the customized groups are correctly mapped into Enterprise portal.

3. on next screen you need to give credentials for Business Connector, you can supply proxy user which we have created for Business Connector.

4. Next screen will ask you to map documents type to document categories in Enterprise portal.

5. On fifth screen you need to configure module documents categories.

6. Next wizard form will ask you to configure document movement, number of days that document need to include in the list of latest documents etc.

7. Configure transaction summaries.

8. By clicking on Next Wizard you will be diverted to Enterprise portal deployments wizard if it is already been deployed and you will be diverted to Manage Enterprise Portal Deployment forum (GO to step 11 else follow order). Check below screen to refer deployment wizard.

9. Next screen will ask you to mentioned IIS virtual directory and SSP site to deploy metadata and EP components. Here you can select option for deployment like if you want to use only webparts then you can select webpart deployment that will enable site to used deployed webpart. By selecting full deployment you will get site templates which will be useful to create site collections under Enterprise portal and pre-generated pages for all system objects. Refer below screen.

10. By click on the last page of the wizard you can finish the deployment as well it will give you two option if you want to restart IIS server after deployment and SharePoint new site wizard (SharePoint admin site).

11. Below you can see Site collection form to create and configure new site. Check at the template section where you can see under the custom template tab that list shows Enterprise portal templates which we can use to create portals.

12. Cross your figures and it everything went well in previous steps then you will find the below screen as a success. :-)

13. If the user you are accessing is not registered or mapped under the Axapta then you will see below message.

14. You can setup the relationship between web user and Axapta user by user relationship form (Administration -> Setup -> User relations). Map the user to correct Axapta user.

15. You can also configure site URL to internal website group.

16. Bingo... Now it's done... You have now setup the site and portal.

This article mentioned to create website portal in general scenario you might need to change or perform more steps in order to deploy enterprise portal in working order in different scenarios and needs.

The default deployment will give you all pre developed web solution and interactive pages inside portal. Obliviously every organization needs customization in portal in order to streamline the Axapta main installation customization to streamline portal and native implementation. It is very important to understand Ax, X++, AOT, AOS and development interfaces inside Axapta client to make that customization possible. Main components need to incorporate for streamlining such customization are changes in Data dictionary, functional default processes and data ownership and security. Changes and implementation in native Ax rollout are not automatically going to enterprise portal. You must ensure the same customizations, changes are been implemented in web solution and components and they both went live on the same point of time otherwise it could create complications and integrity issues with present and new data. As within this article I am not covering such practices but given high level idea before any customization with portal.

Customization of Enterprise portal in Axapta.
1. Below you can see the AOT (Application Object Tree), AOT displays entire object model of Axapta like classes, objects, forms, website (web forms, weblets, menus, urls etc...). You can directly add, update and delete that object and start customizing the product from AOT only. But it is not advisable, it's always advisable to take backup (export *.aox) of those target models and object which we are planning to change. Then using projects window you can add new forms and customization which will give you better control over changes and customization.

2. After customization you can compile project or root element of AOT Tree (if doing direct level customization), refer above image which shows in bottom pane about compilation results.

3. After compilation, changes should deploy through manage deployment form (Administration -> Setup -> Internet -> Enterprise Portal -> Manage deployment). IT will give you the control about which associated and register site should receive the recent updates. Unfortunately there is no such versioning and source control available for such changes and deployment, so always it's good practice to export customizations and take the backup and archive those export files (*.xoa). By clicking update button on the screen it will deploy the changes on the sleeted servers and will show deployment summary in new window as seen in below screen.

4. After deployment you can find such customization (new menus, urls, links, and form etc objects) in the list properties of different Enterprise portal’s webparts.

Here in the article, I haven't gone deep inside about the customization. Will defiantly try to come-up with new detail article about the customization of the Enterprise portal.

Hope my article helps you in any manner to understand the process, please share your views and send your feedback to nilayparikh@gmail.com

Call outs (Plug-in)/CRM Adapter Vs Ajax Calls for Microsoft CRM integration

2009-08-01T18:30:00.000+05:30

Recently, I have been undergone with experimental investigation during my CRM exploration drive with CRM 4.0. To design event driven data receive from Microsoft CRM, we can use some of in-house functionalities like Workflow, Callouts (Plug-in for 4.0 Version) and BizTalk (CRM Adapter/SOAP Adapter - CRM Web Services) but with these all methodology we are creating extra overhead on CRM Server only. I trial with my laptop and VS 2008 to create such scenario and compare results between different implementation methodologies and I found very interesting results.

My System Configuration is Turion x2 64 1.9 GHz with 4 GB RAM, this system I have used for the excremental drive. I have dedicated virtual PC to CRM and bombard through VS 2008 test agents and generate real-time scenario of 10 active users which I can say equivalent to standard production servers configuration and 100 active users for any SMB organization.

Workflows / Plug-in(s) & Callout(s):

I found some interesting results, first let me go through workflow options, workflows are basically design over the land of Windows Workflow foundation and they follow the same process like custom plug-in(s). I tried to integrate different entities and their data using event driven BizTalk SOAP calls. Workflows have very high latency (more then 4 minutes approx) and also they are very resource consuming, I found my processor was used more then average 65% and some time it picked 100 % utilization. It's oblivious that workflow will be slower as CRM itself takes 30-60 sec to invoke workflow after an event as well as workflows have been called by in process .net assembly (similar like call out assembly) so it would be slower and resource consuming option.
Advantages:
- More control over the Integration.
- Stronger security.
- Collaboration integration.
- Custom composite messages and control over message build
- Synchronized and asynchronized approach, gives flexibility.
Disadvantages:
- Higher latency.
- Resource consuming.
- Slow

CRM Adapter:

CRM adapter is general practice for integrating Microsoft Dynamics CRM to external world. It is quite easy and quicker to building solutions. It allows you to fetch all entities in the standard schema format as they have in Microsoft CRM. Let's back to performance, as it relay on different mechanism then workflows and callouts, it talk with SOAP services (Web Services) in CRM so it's bit quicker the previous option but it also generate process overhead on CRM server/databases to fetch data. The same amount of resource utilization i observed for CRM adapter.
Advantages:
- Easy to design and develop schemas and messages.
- Quick development.
- Faster then workflow(s) and plug-in/(s)callout(s).
Disadvantages:
- No control over message structure.
- Only support asynchronize method.
- Resource consuming.

Ajax Calls (Java Script calls):

'AJAX' is very fancy term but believe me it has same qualities inside implementation. Asynchronized JavaScript and XML (AJAX) generally used into web development but it have very large scope of implementation across various scenarios like such client face integration to Servers. I would like to elaborate such functionalities I have implemented in past and the result with this excremental drive. Supremely, it works really well with such load and handling such great traffic of requests without any failures and heavy latency.

img. 1. AJAX WS* Call

In above code you can write the synchronize post method as well as asynchronize post method, as in my organization we are using only IE for CRM client such Microsoft active x object works well with it.
Advantages:
- Very quick as it been called by client side code.
- No overhead for integration on CRM server.
- No read call back to database and CRM Server (unless you require to fatch GUID PK, still they are manageable with advance coding in java script)
- Full control over the message and schema, you can build custom schemas and message as per organization need.
- Full control over process, it support synchronize and asynchronize methods.
- Easily we can develop such bespoke services to plug into existing infrastructure.
- BizTalk and other Integration tools can easily connect without any third party components or services as WS* are standards.
Disadvantages:
- Security, need to provide enhanced security features, if it's called and manage by internal domain then it would be not issue.

Personally, I believe all methods and technologies have advantages and disadvantages; it's up to us what we select as best for the scenario.

Hope you like this article, please give your feedback to nilayparikh@gmail.com

How to get row counts from Recordset variable in SSIS?

2009-03-18T22:03:00.000+05:30

Not straight forward but not difficult,
Get data into recordset destination, which store in SSIS variable.

Recordset is native object (not managed object) and SSIS don't come up with any component which counts the row number from in memory storage. You can read it and map to another variable only using Script Component.

Once you get the count value in variable, now it's up to you how you want to innovate the use... e.g.

Scaling out MOSS 2007 implementation using Windows network load balance clusters (NLB)

2008-10-16T17:35:00.000+05:30

Scaling out an existing implementation or implementing high available performance clusters are always challenge, I have hammered my head many time with the requirement with different Microsoft product like BizTalk, SQL Server, MOSS 2007, SharePoint 2003 etc. From my experience I can say to scaling out MOSS 2007 implementation or implementing high performance cluster is comparatively easier then other MicroSoft products. Well if we talk about performance, so I would say it's majorly depend on two things, back end SQL Server is the first and WFE (Web Front-End) is the second. There are some other stuff which are also important for achieving high performance and availability in implementation but here I would like to mainly focus about WFE and MOSS 2007 Farm implementation on Windows network load balance (NLB) cluster in this post.

To demonstrate, I have used two Windows Server 2003 on single NLB cluster. I would also like to discuss some other windows features which are very helpful in such implementation like network connectivity and physical connections, IIS, NLB manager tool.

Network connectivity and physical network design is very important to achieve best of it, network between servers should be very fast and high available and network design should have minimum possible latency between two servers.
To get best out of IIS 6.0, we can implement compression stream, it also improve performance to some extend.
If the physical servers haven't balanced configuration then I would prefer to configure priority and also set distribution bandwidth according to amount of performance excellence between them by using NLB manager. If WFE servers across your farms are not running smiler services or not having same configuration then users or requests can experience imbalance depend on the server which is serving them.

NLB Manager

Host Properties

You can see above how to configure load distribution between servers on specific ports if requires.

Well, now we are all set to implement MOSS 2007 in Windows Server 2003 NLB environment.

I have installed Office Server 2007 on both node and selected all component to install, here it's up to need, weather you are planning to dedicate the server for WFE or not (Which will be the requirement in real world scenarios).

That's it, what am I read "That's it"!!!, yes you are right, you just need to install on NLB nodes, another important thing is load balanced URL, I have already made an entry into DNS server with NLB domain and portal URL with NLB virtual IP address, so when I create portal (SharePoint application), it can resolve header and translate name address to virtual IP.

DNS Manager

Now there could be two questions, if this is fresh installation then how to create SharePoint application on load balance or if it's scale out installation then how to extend the existing application. Not a big deal, below are the answers.

How to create new SharePoint application on NLB node?

1. Go to create application page using Application Management in Central Administration Site.

2. Configure the application as standard new SharePoint application. But make sure you are providing the DNS entry we have made for load balanced URL (which translate the virtual NLB cluster IP).

3. After creating application you can notice all load balanced IIS servers are been populated with new SharePoint application. All IIS node servers now have physical website on separate location and respective entry in IIS website.

Bingo!!!!, we are done, now you are on load balance application, let me access the application from different client machine and let's check IIS logs for the websites that which servers are serving our requests.

WIN 1 server's log:

WIN 2 server's log:

You can see above logs' snapshots and see randomly both servers are serving our HTTP GET/POST requests.

We are ON SPOT!! Again BINGO!!

Now the second question, How to extend existing SharePoint application on scaled out NLB Mode?

A very simple answer for that question, you can extend the existing SharePoint application by extending web application to another IIS application from Application Management in Central Administration portal. Make sure you use load balanced URL for the same as we have used in above example.

Hope you like this article, please give your feedback to nilayparikh@gmail.com

Install SQL Server in Windows Cluster environment as failover cluster.

2008-10-02T18:22:00.000+05:30

Last few weeks working on building custom cluster environments and custom applications which can support cluster. So took that chance to take some snapshots of SQL Server cluster installation. As other MicroSoft product, SQL Server Cluster installation is very much easy and simple. Let's have a look at below procedures need to follow while installing SQL Server in Windows Cluster Environment. I setup my windows cluster environment from use of Virtual Server 2005 and build two Windows environments and took them both in cluster. Windows cluster configuration is not in the scope of the this article. I will surely produce the article for Windows Cluster too. But for this article I would like to focus on SQL Server 2005 Cluster environment.

The same standard SQL Server installer allows you to set up the SQL Cluster, but if your host OS is Windows Cluster then it will enable few options while installing service for enabling failover cluster for SQL Server Database Engine and Analysis service, no other service support clustering. So let's see below steps to install Database Engine as failover cluster.

1. Follow the standard SQL Server Installation steps but at the screen while you are selecting components to install or upgrade you will notice one check box option labeling "Create a SQL Server failover cluster" for SQL Server Database Service and Analysis Service only, as these both service only support failover clustering and cluster implementation. Please have a look to the below screen, you must check the SQL Server failover cluster option to continue installation or upgrade in cluster mode.

2. If you haven't checked box for cluster installation or upgrading then it following screen will be standard installation installation wizard screen, else you will see the below screen in order to setup the cluster. Every cluster need the virtual server name, that virtual name behave mostly like named instance, you can provide the virtual server name while setting up connection strings to connect databases or while managing cluster through management studio. Please make sure few important checks before naming the SQL Cluster, to avoid conflicts in name and network resolution please provide unique network object name (like computer name, host names etc...). Refer below screen.

3. The next wizard screen is very important to configure the cluster server's virtualization configuration. The Virtual Server Configuration screen will allow you to setup virtual load balance IPs for the networks (These networks are only cluster virtual network which generally we configure while setting up Windows cluster only). You can setup load balance virtual IP for those networks, I will surely suggest you to create public and private virtual IP separately and avoid heartbeat network channel for the cluster for the performance benefits. Refer the below two screen.

4. The next screen will allow you to add the resource to respective manageable cluster group, in my setup I have just created one cluster group so there is not much choice for me to select :-) but it's a really good practice to manage proper cluster group for better and clean administration. Another important input the screen show is data file's location, it will enumerates the physical drives you have added as the resources in cluster. Ideally the drive should be accessible and shared among all cluster servers and should have valid and proper permissions for required operations. Ideally it should be SAN or SCSI device. Also would advice to cross check after installation that physical drive resource has been added to dependencies in SQL Server resource.

5. The next screen will be for configuring Cluster Node. It will allow you to select cluster nodes. It was my first cluster node installation so just displaying single node in list. Other node you can notice under the label "required node".

6. BINGO.... You are back to main land, and standard installation starts, the following steps would be smiler then regular non cluster installation.

SQL Cluster installation is very simple to install and also to manage. I hope the article helps you to install it first time on your machine. Will try to come up with new articles on clusters and installation. I recently setup my R&D environments in clusters and enjoying to working on it. Next installation will be MOSS 2007 on cluster, BizTalk 2006 R2 installation. Will come up with those articles soon.

Hope you like this article, please give your feedback to nilayparikh@gmail.com

System Center Capacity Planner 2007 - A nice point to kick-off IT Infrastructure planning and design for MOSS 2007

2008-09-23T01:20:00.000+05:30

System Center Capacity Planner 2007 - A nice point to kick-off IT Infrastructure planning and design for Microsoft Office SharePoint Server 2007 (MOSS 2007).

Recently, I have been trying and exploring number of best practices to design and plan IT Infrastructure especially for BizTalk and SharePoint Portal Server implementation. In my exploration drive I come across a very nice tool called System Center Capacity Planner 2007. Here in the post I would like to focus just on MOSS 2007.

The tool allows you to plan hardware, servers and service across organization, different location, etc. There nice demo and help already available inside tool itself so would not like to paste here but there are some snapshot, I would like to post along with the blog post.

You can download the tool from below link. http://www.microsoft.com/downloadS/details.aspx?familyid=DBEE0227-D4F7-48F8-85F0-E71493B2FD87&displaylang=en

The tool also provides nice wizard to start planning. Have a look to some below snapshots for designs and reports.

1. Global Topology View

I tried to build organization's global topology. Where server farm is located in one continental and that is connected to various offices, branches across the glob. To evaluate such need for implementation, hardware design, scale out processes for services, service and load balance distributions are tricky and time consuming. System Center Planner 2007 perfect tool for such analysis and design. It simulate load and provide stats which could be very helpful for such design and planning.

2. Site Topology View.

3. Server Role Configuration, to design scale out topology it's very important. Also help while making decision to buy new hardware and scale up/out planning.

4. User Profiles which are role based and component base. Allows tool to predict more accurately.

5. Simulation Report.

Tool generate rich statistics in report, where you can estimate with such number of users and dependent on their role, latency in network what will be longest transactions, I/O utilization, Highest CPU Utilization, Bottleneck analysis, Highest storage space utilization, connectivity utilization, etc. It also generate different stats for all servers, clients, SANs, WANs and Thresholds.

Please send your views and feedback at nilayparikh@gmail.com

Thanks.

Microsoft Dynamics Ax 4.0 and Integration Methods (Part-1)

2008-08-10T18:47:00.003+05:30

Microsoft Dynamics Ax 4.0 and Integration Methods (Part-1 WebServices -
Application Integration Framework, AIF)

My Dynamics Discovery :-) (nice term) reached to bank of the river "AXAPTA",
few days ago ... Well I have worked for Axapta in a very small project but
long time ago and with previous version too. So it's my turn to refresh and
edge my knowledge on technology Ax. Blogging is my hobby and I can stop my
self to write the first post of the series.

These days if something is making me exciting is Microsoft Dynamics and bet
this is the most hottiee topic these days among Microsoft Professionals. Let's come to the technical subject. Axapta support many integration methodologies to get Integrated across the platforms, like BizTalk Adapter (Axapta), WebServices, X++ AOS Classes and .NET / WCF. Here in this part of article I am focusing on AIF WebServices and possible innovation, I humbly invite you to share your views and ideas on this article.

AIF already has very strong and competitive AIF infrastructure to develop and
config functional WebServices which has direct interaction at AOS Level
(Application Logic). To create such AIF WebServices you need can refer below
steps. In step of creating WebService integration point (Action) you must have at least one WebSite configuration to create respective webserice and supportive files. That website should have configured with Windows integrated authentication as it supported by default in web.config, later you can change and reconfigure it.

1. Configuration of WebSite.

2. Adding WebServices inside Actions.

By clicking on generate button, Axapta will populate all webservices, reference assemblies in Bin folder as well as supportive schema documents in web share folder. Axapta only generate 'Microsoft.Dynamics.IntegrationFramework.WebService.dll' assembly in Bin folder and you need to place dependencies like 'Microsoft.Dynamics.IntegrationFramework.WebService.Process.dll' and localization resource assemblies in respective folder of the ISO Standard name in Bin folder. If you are configuring it in default location it will be present there but for another location you need to configure it manually.

3. Review the Web.Config file.

4. Generated files.

5. WebService view.

6. WebService Method view.

Till here the article demonstrate to configure AIF WebServices with default processes. Axapta also support advance customization and on top logic over Axd classes. Here for an example I am going to explore the customization in logic for AxdSalesOrder class. As well as any new Axd sys level class can provide action point and ability to generate such webservices to integrate data between systems. Please have a look to below steps to do advance customization in AIF actions.

7. Code help explorer.

Code help explorer can help and guide you about the classes and diffrent methods they support, by using it you can find out meta information for every action class and their methods as well as cross references.

8. Class tree to explore AxdSalesOrder Class.

9. Edit and customize code logic inside AxdSalesOrder.

You can see above the all implemented logic behind AxdSalesOrder for action methods. You can customize it as per the requirement.

In further to discuss about the AIF WebServices, they are inbound as well as outbound data services, these days when service orientation is on demand and proving it self as very cost efficient, open end and system friendly in such age AIF is big step towards make implementation and organization more service oriented. As WSE* standards are fully compatible with across the table and technologies as well as if I focused on Microsoft only then newly introduce technologies like WCF and WF could be also do well in such domain with Workflow processing and open end secure communication with Axapta. Defiantly web site integration will be a great advantage of the method too. I welcome your comment and views on the thoughts.

Hope you like my article and please give your feedback to nilayparikh@gmail.com

Experience on SSIS and SAP, inspired after viewing some presentation

2008-06-29T18:05:00.002+05:30

Experience on SSIS and SAP, inspired after viewing some presentation.

I inspired for the article from one article posted on MSDN blog about SAP and SSIS. It's really cool stuff from Matt Masson in PowerPoint presentation.

Please have a look into the below presentation by Matt Masson. it's really great to know about SAP and SSIS!

BizTalk Adapter Pack (SAP) installation

BizTalk Adapter Pack (SAP) SSIS integration

BizTalk Adapter Pack (SAP) SSIS integration (Advanced)

Below I am just sharing my experience with SSIS and SAP BW/APO.

I have worked in one of project doing integration between SAP BW/APO modules to other ERPs (like JDE, PeopleSoft etc). For that we used to workout everything by building IDocs inside a SSIS Script Component. It's very time consuming but right now I don't think rather then BizTalk there is any simple option available to build and submit IDocs. I Used to build all require segments like Control, Header and Data inside script task and then composing message through Flat File Connection Manager to build IDoc.

As of my best knowledge currently no tool available in market which can give full functionality to play around IDocs inside SSIS (may be I am wrong, you need to search extensive what such features you are looking for and what are available). But the best option I have implemented almost one year ago in my first SSIS implementation was creating IDocs programmatically inside SSIS. Obviously, I need to implement my logic inside script activity and flat flie connection manager, it make system bit slow but still configurable and acceptable. It is time consuming for initial period to build logic but I think once we build it, it's best option to stick on your requirement with few changes in your code to build new structured IDoc.

Really hoping from Microsoft to have SAP Connection Manager if anything near to automation/parser and connection manager it will be adding great value to product and will be great help for such people don't want to connect to SAP DB directly and want to push data by safer IDoc option.

At end really want to appreciate the efforts from Matt for such presentation.

Thanks for reading my views and please contact me for your feedback or suggestion on nilayparikh@gmail.com.

Looking beyond to ETL from SSIS, next door can be EII - Enterprise Information Integration.

2008-06-26T20:56:00.003+05:30

Below paragraph is pre-face of the article, and it's mentioned about inspiration of the article, if you wish to skip it please go to next paragraph. :-).. But I would recommend reading it.

Recently many things cooking around Microsoft Data Services, I hope it wills surely include more services which can be much helpful to implement difficult EII scenarios. New term Database as a Service (DaaS, name cloning from SaaS). Frankly speaking these days war between terminologies is far higher then technologies, aaahh forget about it, it's very confusing which do what!!!! nowadays, terminologies are becoming face of product and patterns are became abstract :-). Anyways let's come to core part of article, actually I was working on very nice requirement where system need to provide a merged data view from multiple system (heterogeneous sources and also multi-cultured also :(..). Previously we used to do complete data integration (As EAI Implementation) with moderate latency across whole system into one web application which provides grid view and reports; it was very frequently used by users and very important too. But this integration consumes 200GB of database for just viewing and reporting purpose (we keep only 15 days data and for that we have to invest minimum of 250 GB data storage on SAN and other resources for implementation of EAI like BizTalk server, message box database, traffic, bandwidth etc...). It was clearly an requirement of EII implementation, but previously it hadn't been identified as EII requirement and individual implementation, too. But after massive discussion (fights) we were implement that system as EII with use of SSIS (SQL Server Integration Services 2005) only. From that time I had the inspiration of this article. Thanks for reading it.

Back to Business:

Microsoft.SQLServer.Dts.DtsClient name space, I am SSIS developer too, but was not much familiar about its functionalities earlier. As a problem I described in above paragraph we design few complex SSIS packages with heterogeneous data sources and data views, merge joins, lots of shorting, querying, cleansing, etc. Here for demonstration I am unable to saw original design but will follow one sample design I have developed specially for article.

In below design, I am trying to querying to two different data sources which have entirely different databases and connected through one common key and I am trying to generate single dataset for viewing and reporting purposes. You can see below data table snapshots where first figure shows first names of people and second table shows last name of people. Both tables are in different database. Here data sources could be any kind of supportive sources inside SSIS for the concept.

Fig 1. Database Tree (two different databases - test_1, test_2).

Fig 2. Table 1 (Db1- table contain first name information).

Fig 3. Table 2 (Db2 - table contain last name information).

Now have a look to sample SSIS package, just I am doing merge operation and storing into data reader as a destination.

Fig 4. Sample SSIS package.

I have also developed one console .Net application for testing and demonstration purpose to call SSIS package as single point data connection which can provide me processed and integrated data within one single record set. Below is sample snapshot of coding, in real-time solution we used WCF services to provide external domain and other application as an exposer of the EII implementation. But here I am trying to demonstrate through C#.Net Console Application.

Fig 5. Console Application Code (C# Console Application).

Fig 6. Receiving data into one data reader.

Fig 7. Output.

You can see above snapshot of output that I received integrated query's result in single set.

Isn't it good? There are many benefit of such implementation I can say. The major benefit is shrink EAI landscape which will decrease support team size (Cost... Managers are very happy to listen, additional benefit) and we can free resources to better performance of it. Another important benefit is, we can avoid non-require expose of database and our precious data to other system which only need for viewing purpose, we can archive it by providing and implementing parametrise/on demand EII Data Services (SaaD :-D). I don't want to go inside other benefits we can archive from such implementation but defiantly it could turn into beneficial implementation for every organization which has such requirement.

Please share your views and comments as well as if you have any suggesting please free to suggest me on nilayparikh@gmail.com

Thanks for visiting my blog.

MOSS 2007 - Workflows and it exposes to Enterprise Information Integration and Enterprise Application Integration

2008-06-25T21:14:00.003+05:30

I am personally very excited about MOSS 2007 and it's brand new key futures, from last few weeks I am really involving my self to see MOSS's new key futures and it's innovative implementation(s) for those areas intersecting with Integration, rather it's Enterprise Information Integration (EII), Enterprise Application Integration (EAI) or Data Integration (ETL). I have worked in my past with SharePoint Portal Server 2003 and it was really complicated to integrate different user activities/events with across the enterprise through BizTalk and Web services. Always it dragged me to develop more custom components/web parts/web controls and calling web services, transferring information and events through BizTalk server. Looks really complicated but believe me it was much tougher then it looks...

Now at new mission I am trying to find out more flexible integration/Work flow base implementations, here I have developed one sample solution to support my view on the Information Integration within MOSS 2007 Tasks and announcement. I am primarily trying to develop some workflow which can pick announcement which is direct live integration with sample Loan Application Management (any custom application generate application details) through WF (which hosts on Windows Services and connected through WCF for my sample, it's open space for any integration tool provide direct integration with MOSS site like BizTalk ;-) etc...). Let me comeback to sample, now I have developed one pilot Workflow which pick each announcement and generate tasks on SharePoint as well as through WCF/Web services it integration with external applications too. Another Workflow application get invoked on completion event of those Task and automatically generate following to do tasks and assign automatically.

Now these workflow also provide me standards functionalities of Windows Workflow Foundation, like listen, event base routing (you can check my previous articles to know more about event base routing), it's really empower developer to go beyond the classic and traditional boundaries of SharePoint sites to next generation workflow and information oriented collaboration development.

Let's back to sample, workflow1 picks announcement and process it and generate proper Task and other artifices, too. In the process of generating automated task and artefacts it's utilize many EII services to authenticate credit and other historical information of an applicant.

You can see below snap, first activity shape will invoke the workflow on any new announcement arrival or creation. Then the workflow is trying to received data and contract through Get activity and parallel processing other code too. At end of successful processing of whole parallel shape it will create all tasks needed for the announcement and at end it will log whole history. Please have a look at figure-1 and figure-2.

After deployment on to your MOSS Site you can check the workflow status by going to Site Settings->Galleries->Workflows. You can see figure-3 and figure-4 for site navigation and Workflow Collection. Site Collection Workflow will show you the association of the workflow to different activities and In-progress workflows too.

Let's come to another sample workflow of the pilot solution, which watch on every complete event of Task and according to the event it flow the application further for consideration and human approval. Where it also involve it self into request response with Web services and EII Service contract through workflow service. And similarly create further to do events in MOSS site.
Have look to MOSS Site Snaps.

1. Creating announcement for sample Loan Application.

2. You can see below figure announcement has been added to list. It shows 11111 id for loan application.

3. After sometime it was picked by workflow and processed, in solution I have appended string called " Processed By...", you can see here it has updated title of announcement and also added new relevant task for the loan application. During the creating of new task it went though to WCF application and receive back some valuable credit information and rating.

4. Task's detail page, you can see here generated description and information.

5. Once Task has been completed another workflow2 processed it and generated siblings task and to-do also it updated the title of the task (You can track related artifacts by "11111" id).

Such futures of MOSS 2007 make life easy for Integration developer and we can also develop such infrastructure suits MOSS 2007 features which help more easy and quick development for complex definitions.

Previously I was using BizTalk adapter or custom .net code but it always want huge resource to implement such workflow and integration for SharePoint site also it increase high volume of data and security threats while such implementation like workflow really help to develop much abstracted information and service access layers and less resource consuming infrastructure artifacts. I analyze such implementing with workflow pattern and pilot application performance results, it has ability to shirk our transaction volume across BizTalk by 40% compare to previous and 50% better performance with easy to maintain requirement. These are not derived figures but these are my assumption on the standard scenario which I analyzed.

Hope you liked this article and please send your valuable feedback and ideas to nilayparikh@gmail.com

Orchestrated windows communication foundation service (WCF + WF = BizTalk Content Routing)

2008-06-09T15:55:00.002+05:30

These days... I start looking back to interesting .net 3.5 stuff, really impressive. Especially I can say with .net 3.5 sequential workflow and its two data shapes (receive and send) and their promotion to WCF services. Infect this is very good functionality toward integrating more complex collaborated scenarios with different domain application. Previously, it was really hard and time consuming to collaborate using open standards like web services, EAI (BizTalk), MSMQ etc, and it was bit effort consuming as well as contain higher risk on success of the design. Here I feel very nice and good solution towards such complex collaboration scenarios' integration to cross domain.

WCF itself is well functionality talk as open with open standards like WSE 2.0+, Queuing (MSMQ/JMS), .net Remoting, TCP/IP and web HTTP/HTTPs, etc. I must say this deadly combination will transform much complex scenarios in very simple WF/WCF solutions.

I tried to build one very simple collaborative solution. This can receive message through WCF Service which is simple passthrough service and routing incoming messages to WF. I found WF is very much flexible towards handling external events, tracking, hosting (with multi-threading).

I tried to draw one quick diagram which might help to understand the concept.

I also tried and simulate some work load on the sample application with the concept and I got surprised with really great performance. I pumped around 200 messages per second with approx size of 50kb and really it went smooth with some complex resource consuming XSLT transformation and file delivery. Where while building the application I programmed to raise max 5 threads for the transformations and delivering WF services. My correlation and Initializing WFs was working on single instance thread. But I was calling other WFs from main WF to performs transformation and delivery services.

Just thought this would be great to share with MS Developer community. I hope Microsoft is planning to put WF and WCF into future version of BizTalk on the same basis. Now, after success of the POC sample, I am analysing the replacement of content routing from BizTalk Server to WF + WCF and planning shift much load from BizTalk to standalone WF Hosting servers.

Let's see how it may go! I am very hopeful for this implementation and success of it as cost wise and performance wise too.

If anyone is interested in POC code most welcome to request on nilayparikh@gmail.com

Thanks.

Consuming WCF Services in BizTalk Server 2006 R2 - Add generated items (WCF Services)

2008-03-26T21:53:00.003+05:30

I was thinking a long back to come up with this article, and now just found time to write about WCF services and it's implementation with BizTalk Server 2006 R2. I have designed few small solutions in past few months to deal with scenarios like Information Integration and SOA with BizTalk Server R2, where I found the great chance to work upon the WCF adapter series as well as Web Service Factory : Modeling Edition. Both made me big fan of them, it's great to work on both, I am not focusing on WebService Factory modeling in this article but will discuss about WCF services consumption inside BizTalk and few videos might help people to work on them.

I would like to share few images below, by reading this images you can understand the service I have coded for sample, if you are interested in sample code please mail me, and I promise you will find that sample someday in you mail box... when that I don't know. :-) But will try send you soon as possible for me.

The best way I feel to consume WCF services inside BizTalk solution is by using WCF Services Consume Wizard which you can find in Visual Studio Template, this wizard will power you by generating schema and port binding configuration files for your solution based on service descriptions. Not going into deep how this work internally here but focusing on the practice I gone through.

Below image shows Generated Items and Visual Studio Template.

Please have a look to the below Video Presentation where you can find what steps I followed to generate my sample project. Hope it will surely help you in any manner while working with WCF and BizTalk together.

As result of Wizard you will get schema and binding files inside your solution, that schema will be generated on basis of the metadata of the consumed services and it could be used as multipart messages in case of request-response services. Below snaps will show you the BizTalk schema presentation against the .net message contracts and data contracts.

DataContract Source Code:

MessageContract Source Code:

Service Implementation Code:

BizTalk Schema:

BizTalk Test Message I used for my sample:

You can analyze your self the presentation of Inbound message and the Contract representation inside WCF Implementation upon the above snaps. You can consume the generated schema in routine BizTalk solutions and patterns, while deploying such solution you can use generated binding information for such implementation with Endpoint transportation and Behavior information regarding WCF hosting and services. It is very good feature provided by wizard which make our life very easy :-). Let's have a view to below video presentation on the WCF Http and Custom Adapter configuration inside through BizTalk administrator console.

Hope you liked this article and it will give you fair idea about the BizTalk Server 2006 R2 and WCF coupling in Solution, as a system/solution designer point of view I really enjoyed to work with this two made for each other technologies and it provide such ease to workout most complex scenarios regarding service orientation and integration. As well as I realize the solution's capability to work over Internet Integration where security is the major concern and we generally prefers to go with ws-HTTP or HTTP channels for integration, on such scale it can help us to give such secured integration system.

Please provide your feedback about the article at nilayparikh@gmail.com.

Web Service Software Factory - Walk through Video (WCF Implementation)

2008-03-19T23:48:00.001+05:30

Recently, I got very good opportunity to work on WCF Services through WebService Software Factory, it's a really nice tool, I have seen so far to build complex WCF services in very easy manner, no really I means. Yes, there are some limitation of the tool and it's not supporting all features of WCF implementation but you can say.. it's very nice kick off point for your project and code, later you can defiantly customize your code by modifying the auto-generated code itself.

While working on factory, I grabbed opportunity to record my activities through one screen recording software and I would like to share that video among IT Professional community. Please have a look and share you view on such beautiful tool.

Service Modeling & Message Modeling walk through:

Code Generation walk through:

You can also find some more videos on below link.
http://www.codeplex.com/servicefactory

Please send me your feedback and suggestions about the above walk-through on nilayparikh@gmail.com

BizTalk 2006 R2 with PeopleSoft Integration & MicroSoft BizTalk PeopleSoft Adapter Overview

2008-03-14T14:45:00.002+05:30

Recently, I come accords requirement to integrate PeopleSoft ERP implementation within existing ESB. I have worked on many PeopleSoft integration through BizTalk, so just taking opportunity to write something exciting about knowledge I have gathered.

MicroSoft is very much aware about importance of PeopleSoft ERP and it's huge existence in Industry, so BizTalk R2 LOB adapter set has include PeopleSoft adapter which is very useful artifact during our PeopleSoft integration, it facilitate developer in many manners like developing schemas, messages. There are few limitation also we are facing with the adapter but those limitation could be overseen with different kind of implementation patterns and of course we can archive most of requirement what we want to with the adapter. There are also many third party adapters are available for PeopleSoft integration but I have explored more, the only one third party adapter I have worked with is iWay last year, yes it is very impressive but while comparing overall prospective I would defiantly go with R2 LOB. But it purely depend on the requirement and what extend we are looking for integrate the systems.

Microsoft Line of Business PeopleSoft Adapter's Architecture.

LOB PeopleSoft adapter's Prerequisites:
# The Java 2 Platform must be installed on the BizTalk Server that the BizTalk Adapter for PeopleSoft Enterprise is running on.
# The PeopleSoft Java Object Adapter JAR file, psjoa.jar should be copied to a folder that is accessible to the BizTalk Server that the BizTalk Adapter for PeopleSoft Enterprise is running on.

Implementation & Design Patterns:
BizTalk PeopleSoft adapter facilitate your requirement to query and receive data from PeopleSoft ERP also you can do basic data operation like Insert, Update and Delete. But here one thing very clear, but it will not allow live integration from outbound direction (from PeopleSoft). To archive such requirement you can implement services inside PeopleSoft which can push live data feed into any queue or physical location where BizTalk or any external service can monitor feeds with very less latency, so how you can get such live implementation.

I would like to share the architecture, I designed for such live implementation landscape recently. You can view the picturing below.

The above architecture, I designed for the two-way integration with PeopleSoft for our client. Low latency outbound messaging integration can be archived by MSMQ or any other queuing implementation between BizTalk and PeopleSoft. Also many time in different scenario requirement we need information integration between different systems that could be archive through cutting edge technology like WCF / WSE 2.0 where BizTalk R2 provide functionality integrate such services, you can also give term like 'EII' to such implementation. More over by implementing such landscape we are moving towards service orientation and service bus. BizTalk provides a ease to integrate many systems within such implementation, also it leverage extensibility of ERP implementation.

You can also go through below links for further references at MSDN.

1. Tutorial: Using the BizTalk Adapter for PeopleSoft Enterprise to Retrieve Data from PeopleSoft Enterprise
2. Tutorial: Using the BizTalk Adapter for PeopleSoft Enterprise to Write Data to PeopleSoft Enterprise

Please provide your valuable feedback on nilayparikh@gmail.com.

Advance mechanism to handle exceptions for .net component used inside BizTalk solution

2008-03-02T16:16:00.003+05:30

Finally after one month’s big vacation I am writing something technical, few days back my one of good friend came to me and asking me about the exception handling inside BizTalk for those scopes which contain calls .net classes and their methods. Well, I enthused about this article, and also a good chance to document some good practices I have worked for in some previous projects to improve the exception management and automating support routing. The article also includes some videos, hope it will help you to understand my views and idea about this mechanism I am presenting here.

In every system or solution exception handling stands with primary requirement, this article is not elaborating exception handling fundamentals in .net platform but I am focusing towards exception handling in .net component which we are adding as assembly inside BizTalk Solution/Project. Those exceptions handled inside .net assembly then the question comes how we can manage execution flow if exception occurs with orchestration logical flow, need to put flags, variables, calls and then with decision shape we can route files or log it within orchestration, correct. Hummm, it’s really odd solution, it can surely fulfill our purpose but it’s not recommended. This is very heavy on orchestration as well not recommended for runtime performance of BizTalk orchestration. Well, there is always an Generic Exception we can handle inside Orchestration for any scope but it will not give specialized block to handle such exceptions like if I want to handle DivideByZeroException and NullReferanceException in different block and then I want to divert the files at different Send Port so my technical support team can look and appropriately correct them individually and send those back for reprocessing, so I can give my customers better and fast experience with BizTalk support group.

An unhandled exception inside .net class which you can surely catch inside BizTalk Orchestration as Generic Exception. But If you have requirement like you need to execute few statements to compensate and then you also want catch that exception inside BizTalk Orchestration to route the logical execution path. Bit confusing requirement but believe me it’s normal one, there you will question me, why I shouldn’t catch inside BizTalk only and execute all compensate statement inside catch block, really possible but it will be not appropriate for performance, suppose if you handled it inside .net assembly then assembly call, refection will improve run time assembly performance inside BizTalk rather then calling multiple time. Yes then the question, what to do about the logical execution flow inside Orchestration, if the requirement want to route the message to any other branch or to any send port if exception occurs, possible? We can rethrow the exception inside .net assembly’s catch block after the compensation has been executed inside code, rethowed exception will be catch by BizTalk orchestration and we can route logical flow as per the requirement.

Below picture shows the implementation for handling the rethrowed exception inside .net assembly.

You can also go through the below videos shows the complete implementation about what I have discussed.

.net Assembly implementation video

BizTalk implementation video

Hope you liked my article and might it will help you, Your valuable feedback and suggestion are welcome to nilayparikh@gmail.com

Enterprise Information Integration - A First Look

2008-01-21T02:00:00.000+05:30

Nowadays, A trend I noticed in IT Professionals, if any technology start with E** will be on demand, and it’s not absolutely wrong because we have seen in last one decade, “E” runs Enterprises. We have live examples of EAI (Enterprise Application Integration), EDI (Electronic Data Interchange), ETL (Extract Transform Load), ERP (Enterprise Resource Planning), ESB (Enterprise Service Bus), and I can bet about EII (Enterprise Information Integration) that would be not an exception among the group of “E”-Technologies. In this competitive edge, every enterprise wants to best data infrastructure with cost efficiency, and that’s bet “innovations never going to end for this requirement”. In this globalization most of companies are operating form different countries with many cultures, people, languages, and these all brings different applications, different vendors, different standards, different platforms, different technology and systems’ cultures. Now, these things don’t making life easy for CIOs, and they makes our life tuff [:-)] like universal food chain (a big fish eat a small one). Generally to handle such scenarios we are implementing EAI, ETL solutions for a low latency/live data integration and historical data integration respectively but still one unnoticed scenarios need some dedicated effort from the technology community these days which is Information Integration, generally we are fulfilling the purpose of Information Integration by data integration and EAI mostly but such implementations will be not cost competitive compare to dedicated EII implementation because EII is gaining traction for enabling data integration without the need for the physical instantiation of the integration. In other words, EII adds integrated reporting capabilities while minimizing impact on existing systems. But before deciding the integration pattern we must have clear idea what we should integrate with EII patterns and what shouldn’t.

Let’s have look to some situations where EII can be ideal and cost/effort effective solution for any enterprise.

Connecting structured data with unstructured data, this can take advantage of EII’s capability of leaving data in place that could not dramatically increase overall storage requirements. (this you can also count as disadvantage of EAI/ETL solutions in such Information Integration scenarios)
When immediate data change required in response to the data view. (This requirement will be not easy and cost competitive for EAI/ETL).
Some operational and regulatory reporting where the data needed is not completely integrated in one place.
When data transformation is relatively light or nonexistent and just getting the data together for integrated query is the biggest challenge.

At bottom line I can state EII is for implementing such scenarios like integrated queries, connecting data without increasing storage requirements and low latency responses.

So, what could be definition of “EII”, I have referred one definition constructed by “Integration Consortium” and I would like to share with you all, EII is the integration of data from multiple systems into a unified, consistent and accurate representation geared toward the viewing and manipulation of the data. Data is aggregated, restructured and relabeled (if necessary) and presented to the user.

Integration Type	Data	Purpose	Audience
Data Integration (ETL)	Historical	Trend Analysis	Decision-Makers
Application Integration	Live Data	Synchronization	IT Organization
Information Integration	Live	Productivity	End Users

Now after understanding of the EII, the oblivious question would come in mind about implementation and the instrument (tool). Many of technology vendors have already initiated and introduce some of tools focusing or additional functionalities in existing tools which support such EII implementations. The instruments like SOA, Web Services, EAI / ETL tools could be the best picks for implementing such requirements. I believe the following requirements are the key of any EII implementation.

Service Oriented Architecture
Mata Data Management
Semantic Information Model
Dynamic Aggregation

The above key topics I believe will be known to any Integration Architect, so any such program or tool can handle or facilitate the above keys to implement in aggregated manner could be used to impermanent EII solutions.

For BizTalk Server, I see very bright scope for such kind of implementation with other coupled Microsoft Technologies like Windows Communication Foundation, Windows Workflow Foundation, MS SSO, Host Integration Server, SQL Server Analysis Services, and SQL Server Integration Services. Additionally the innovative steps like ESB by Microsoft Patterns and Practices are giving very high level of confidence to such implementation through Microsoft Platform.

If I talk about my experience with EII system then yes, with the grace of God I got such precious opportunity to put the first EII implementation in GO-LIVE state at my organization.

I will try to come up with more articles on such immerging Enterprise Information Integration with BizTalk and Microsoft Technology specific articles. Believe me there is lots of things to share with you about my exciting experience for Enterprise Information Integration.

Thanks for BEAR me during the article. Your valuable feedback and suggestion are welcome to nilayparikh@gmail.com

BizTalk 2006 / R2 Publishing Throttling State #2 (imbalanced message publishing rate, input rate exceeds output rate)

2008-01-16T00:18:00.000+05:30

Our next destination in BizTalk Throttling exploration drive is inbound Throttling states, publishing throttling state #2 Throttling due to imbalanced message publishing rate (input rate exceeds output rate).

Throttling state # 2, due to imbalanced message publishing rate, input rate exceeds output rate:

This is the most simple scenario to understand, here imbalanced message publishing rate means the ratio of the incoming messages by outgoing messages or vice versa.

The following is the condition of the Throttling state:
- (Message publishing incoming rate / Message publishing outgoing rate) * Rate overdrive factor in %

This condition belong to rate base throttling, For inbound (published) messages, BizTalk Server throttles publishing of messages if The Message publishing incoming rate for the host instance exceeds the Message publishing outgoing rate * the specified Rate overdrive factor (percent) value. The Rate overdrive factor (percent) parameter is configurable on the Message Publishing Throttling Settings dialog box. Rate based throttling for inbound messages is accomplished primarily by inducing a delay before publishing the batch of messages into the MessageBox database. No other action is taken to accomplish rate based throttling for inbound messages.

In following general scenario, I found the the throttling state can be possible in environment.

- very high demand to process messages and less available resources to process those messages.
- high use of co-related messages and self-correlated ports, and multiple subscribers for single received message.
- Slow outbound adapters.
- imbalance between host configured for inbound adapter and host configuration for XLANG / host configuration for outbound adapter.
- high processing complexity.

Understanding about Publishing throttling threshold parameter,

Minimum number of samples: Minimum number of messages BizTalk Server will sample for the Sampling window duration before considering rate-based throttling. If the actual number of samples in a sampling window fall below this value then the samples are discarded and throttling is not applied. This value should be consistent with a rate at which messages can be published under a medium load. For example, if your system is expected to handle 1,000 documents per second under a medium load, then this parameter should be set to 1,000 * Sample window duration in seconds (or more precisely, 1 * Sample window duration (milliseconds)). If the value is set too low, then the system may experience a throttling condition under low load. If the value is set too high, then there may not be enough samples for this technique to be effective.
Enter a value of zero to disable rate based inbound throttling.
The default value is 100.

Sampling window duration (milliseconds): The time-window measured in milliseconds, which is used to calculate the publishing rate based on the samples collected. The duration should be increased if the latency required for publishing a single message is high.
Enter a value of zero to disable rate based inbound throttling.
The default value is 15,000.

Rate overdrive factor (percent): This controls how much higher you allow the request rate to be than the completion rate before a throttling condition occurs. For example, if messages are being published at a rate of 200 per second and this parameter is set to 125, then the system will allow the publication of up to 250 messages per second (125% * 200 = 250) before applying throttling. Specifying too small a value for this parameter will cause the system to throttle more aggressively and could lead to over-throttling. Specifying too large a value for this parameter will cause under throttling and prevent the throttling mechanism from recognizing a legitimate throttling condition.
The default value is 125.

Maximum throttling delay (milliseconds): This is the maximum delay BizTalk Server will impose on a message instance due to throttling. The actual delay depends on the severity of the throttling condition.
Enter a value of zero to disable inbound throttling.
The default value is 300,000.

Triggering Mechanism of State #2 publishing throttling: In the sampling window duration, if the incoming/outgoing ratio excceds the configured parameter "Rate overdrive factor" value then BizTalk will be in throttling state.

Throttling Action: Block the publishing thread for a dynamically computed time period until the Message Publishing Incoming Rate is at par with the Message Publishing Outgoing Rate * the specified Rate overdrive factor (percent) value. By blocking the publishing thread, BizTalk can reduce the queued up messages, I have noticed many time the queued up inbound messages has very bad impact on the memory as well as the processing on the BizTalk artifacts. And the impact of the queued up messages is showing exponential memory & utilization which can also impact indirectly overall to every processing in BizTalk. So by blocking publishing thread, BizTalk is reducing incoming message flow and keeping flow healthy.

Sample collection of BizTalk Perfmon data and Graph: You can view in below graph that when the publishing throttling rate condition is setisfying the system in throttling mode #2, also you can analyze as per the sample window state through it.

If you would like to have the perfmon logs data, biztalk throttling configuration snap and graph in compress file for studying or analyzing it further for study which I have simulate for study, please write me to nilayparikh@gmail.com

Hope this article helped you to understand state#2 of throttling for you.

Related articles posted by me on the same blog:
BizTalk 2006 R2 - Throttling - Perfmon Parameters - My Experience (Tuesday, January 15, 2008)
BizTalk Server 2006 / R2 Throttling Mechanisms (Friday, January 4, 2008)

MSDN Resource Link:
http://msdn2.microsoft.com/en-us/library/aa559591.aspx
http://msdn2.microsoft.com/en-us/library/aa559893.aspx
http://msdn2.microsoft.com/en-us/library/aa559628.aspx
http://msdn2.microsoft.com/en-us/library/aa578302.aspx
http://msdn2.microsoft.com/en-us/library/aa547859.aspx

Thanks for BEAR me during the article. Your valuable feedback and suggestion are welcome to nilayparikh@gmail.com

BizTalk 2006 R2 - Throttling - Perfmon Parameters - My Experience

2008-01-15T03:19:00.000+05:30

BREAKING NEWS:These days.. Dr. Time Pass is spending most of time to understand and implement best throttling scenarios :-).

Let me come back to official language. Few day back, I saw lots of throttling in our BizTalk environment. My enthusiasm drag me to the further exploration in throttling mechanism. Let me grab opportunity to document some cooked things.

Since last weeks, I saw my BizTalk environment is facing suddenly unusual throttling behaviors so I tried to find the root cause behind the throttling state in my BizTalk environment.

The environment is facing lots of message delivery throttling during pick processing hours, and due to the delivery throttling state, and delays our whole environment get stunned. After this incidence, I start taking this functionality very seriously and now digging lots of stuff related to it, here in this article I am going to share my experience as well as the understanding I got after going through many books, articles and MS-KBs.

As we all BizTalk professional aware about the inbound message processing and the internal BizTalk instruments takes care about the inbound message management. In short if I like to explain the word "inbound message" then I can stat "messages towards messagebox" as inbound messages, in BizTalk messagebox is the heart of any environment or implementation. We were facing problem due to unsymmetrical messages' incoming publishing rate and messages' outgoing publishing rate, I noticed in our environment incoming message rate is very high in compare to outgoing message (here outgoing means subscription to messagebox, we are talking about deliveries), in resultant the number was increasing for the messages in memory. BizTalk recognized and our environment moves towards throttling state but still throttling was also not able to make the system stable and put it again towards stable node. Due to the very heavy message rush and less outgoing rate very high number of the messages were waiting in queue for processing. We saw almost 300-700 seconds as delay time during the processing. And finally the nightmare came to true :-0, and counter called "Days without Servility 1" need to reset. The thing I would like to convey is the requirement of the optimized configuration parameter for the throttling while setting up any host. I will stat 'must' recommend for setting-up a set of well configured and planned host for the pick time processing for the critical interfaces. I don't want to drive my article into implementation designing, but I would like to come back to the throttling scenarios.

In this article I would like to focus around perfmon counters, few important ratios, throttling states.

Let's dig first to some state counters, state counters shows the state of the current situation in your environment under monitor. BizTalk 2006/R2 comes with very useful perfmon state counters.

some high (*) counters like High database session, High database size, High in-process message count, High message delivery rate, High process memory, High system memory, High thread count, these all counters represents only a state by showing value like 0 or 1, where 1 the respective focused area cross the configured or warning level. These shows very useful information which should be monitor in your Biztalk environment.

In-process message count, Active instance count are two message queue counters which show the Number of in-memory messages delivered to the XLANG engine or the outbound messaging engine that are not yet processed and instances are in active state in memory referring to the EPM or XLANG engine, respectively.

Specialized message delivery counters like Message delivery delay (ms), Message delivery incoming rate, Message delivery outgoing rate, Message delivery throttling state, Message delivery throttling state duration, Message delivery throttling user override. These all perfmon counters should be monitored regularly during the pick processing to understand the behavior of your environment and the coming out patterns to understand and prevent the failures.

Message delivery delay (ms): The current delay in ms imposed on each message publishing batch (applicable if the message publishing is being throttled and if the batch is not exempted from throttling).

Message publishing incoming rate: Number of messages per second that are being sent to the database for publishing in the given sample interval.

Message publishing outgoing rate: Number of messages per second that are actually published in the database in the given sample interval.

Message publishing throttling state:

0: Not throttling
2: Throttling due to imbalanced message publishing rate (input rate exceeds output rate)
4: Throttling due to process memory pressure
5: Throttling due to system memory pressure
6: Throttling due to database growth
8: Throttling due to high session count
9: Throttling due to high thread count
11: Throttling due to user override on publishing

Message publishing throttling state duration: Seconds since the system entered this state. If the host is throttling, how long it has been throttling; if it is not throttling, how long since throttling was applied.

I would like to place basic example of some perfmon data and counters in graph representation.

I will try to explore on Message publishing throttling states on my coming articles.

Thanks for BEAR me during the article. Your valuable feedback and suggestion are welcome to nilayparikh@gmail.com

Related articles posted by on the same blogs.
1. BizTalk Server 2006 / R2 Throttling Mechanisms

BizTalk Server 2006 / R2 Throttling Mechanisms

2008-01-04T17:36:00.000+05:30

BizTalk 2006 / R2 comes with good relief in terms of performance and throttling mechanism, if you work close to administrating and performance tuning job for BizTalk environment or any messaging system then you can visualize the complexity of pick processing time scenarios in any messaging system. Here with the article I am focusing to the particular BizTalk throttling scenarios and their solution. I have derived below contact by referring few books as well as my personal experience while working on performance criteria for BizTalk environment.

The below cases might not cover all the possibilities throttling scenarios of live BizTalk environment, but I have tried to collect as much from my memory.

Case: High message ratio for delivery rate / completion rate

The higher side of the ratio value indicates the service is unable to handle the rate of inbound/incoming messages to the environment. As solution we can throttle message delivery and make available as much resource as possible to BizTalk Server artifacts like XLANG and outbound transports. As a result of improved message completion rate, message latency rate and IO operation rate for message box will be reduced exponentially, also we can get more benefit by cleaning up the queue at out-bound end point.

Ideally environment should maintain ratio value around 1.05 – 1.07.

We can get monitor message delivery rate and message completion rate through various instruments like WMI / OM object / Perfmon counters / etc.

Case: High publishing ratio for request rate / completion rate

High ratio is indicates of the message box being unable to cope up with the load. To improve such scenario in our environment probably we can block publishing threads to slow down the rate and also we can indicate the service class to slow down the message-publishing rate. In the case there is no such solution available apart of scaling out the SQL Server environment where message box has been deployed, if the environment is facing such problems concurrently then it will be appropriate decision to scale-out the SQL Server environment.

I don’t have accurate observation for the value of the ratio, but I am sure that every organization must need 1 to called reliable messaging services :-).

To get the ratio value, we need to monitor entry and exit of commit batch call.

Case: Process memory exceeds a threshold

If process memory exceeds a threshold then probably your environment and batch process need steep memory, if you are getting such alerts or messages regularly then you should think about the steep memory for the environment. In this scenario, it has affect components like XLANG and all kind of transportation (adapters/pipelines). It also indicates services to dehydrate and shrink catch.

By monitoring private bytes of the process we can get clear idea about the memory threshold for the individual processes. In BizTalk hosts can be treat as processes.

Case: System memory exceeds a threshold

Same possibilities we can assume as we discussed for case “Process memory exceeds a threshold” above.

Here you can monitor the physical memory and trend through perfmon.

Case: Database session being used by the process exceed a threshold count

You can throttle publishing inside the BizTalk environment. To improve such condition, we need to tune the SQL Server Database properly and need to scale up/out as per our threshold requirement. In this scenario XLANG and all inbound transports will get affected. There is not such quick resolution available for the scenario but we can reduce/block the external/idle sessions available on SQL Server. If BAM Notification / BAS are configured to your environment and their database are also running on same server then you can disable / suspend those less priorities services to resume the message processing to normal.

Such scenario can affect XLANG objects as well as all type of inbound transports.

You can monitor session on message box by Perfmon counter / SQL Server Counters / Management Studio / etc. For cluster environment you can monitor session per message box.

Case: Process thread count exceeds a particular threshold

Throttle publishing, delivery and it indicates to reduce the thread pool size.

Such scenario affect XLANG objects as well as all kind of transports (adapters / pipeline). There is no direct solution for the scenario, long term solution could be scale-out the server box and suggestion will be to migrate 64x or itanium (or equivalent) processors server boxes. To handle some pick processing time during the day, we can provide more thread and priority to the host processes handling the threshold.

You can monitor count like threads per CPU to undertake such scenarios / cases.

BizTalk Server 2006 / R2 is handling such throttling automatically but we can get much improved performance by giving some favor to the mechanism. To perform auto throttling BizTalk Server uses the configuration parameters, I will try to come up with detail review of each parameter in next coming article.

Thanks for BEAR me during the article. Your valuable feedback and suggestion are welcome to nilayparikh@gmail.com.

BizTalk Custom Pipeline Components - Framework Overview Part-1

2007-12-26T20:17:00.001+05:30

In this article I am trying to put consolidated information for developing custom pipeline components. Hope you like my efforts; frankly, there is nothing new in the article but it's all together.

Receive and Send pipeline performs below basic operations in BizTalk Solution,
- Encrypt outgoing and decrypt income message
- Handle digitally signed messages and verify them
- Validation
- Handling promotion
- Disassembling and assembling coming or processed messages.
- Wrapping header and footer into messages.

The below I have listed components and their jobs in BizTalk Solution.

Pipeline Components	Stages Employed	Tasks
General Component	Decode, Encode, Pre-assemble, Resolve Party or Validate	Take one message process message and produce zero or one message
Disassemble Component	Disassemble	Split message, promote custom properties
Assemble Component	Assemble	Used to wrap message with head or trailer or both
Probe Component	This is not an independent component. Any pipeline component can implement the IProbeMessage interface if it must support message-probing functionality.	Enables the component to check the beginning part of the message

To develop general pipeline component we can use following interfaces. These are coming as part of SDK and BizTalk Solution (BTS).

IBaseComponent Interface
IComponentUI Interface
IComponent Interface
IpersistPropertyBag

IBaseComponent Interface

Members	Usage
Description	Property. Used to specify small description about pipeline component. Description is visible on pipeline properties page at design time.
Name	Property used to specify name of pipeline component. Name is visible on pipeline properties page at design time.
Version	Property used to specify version (example 1.0.0.0) of pipeline component. Visible on pipeline properties page at design time.

IComponentUI Interface

Members	Usage
Icon	Property used to provide icon associated with pipeline component.
Validate	Method. Pipeline designer before pipeline compilation to verify that all configuration properties are correctly set calls this.

IComponent Interface

Members	Usage
Execute	Method. Does specific processing/massaging in inbound message and produces output message to be forwarded to next stages of pipeline or message box.

IpersistPropertyBag

Members	Usage
GetClassID	Method. Retrieves the component's globally unique identifying value.
InitNew	Method. Initializes any objects needed by the component to use the persisted properties.
Load	Method. Used to load property from property bag.
Save	Method. Used to save property-to-property bag.

I will surly come back with more detail articles and implementation with sample code for the subject.

Thanks for visiting my blog and please send your feedback or queries to nilayparikh@gmail.com.

Cheers!
Have a nice day!

Recoverable Interchange Processing - BizTalk Server Video Tutorial

2007-12-26T00:52:00.000+05:30

Again I am coming with very good video demonstration by Stephen W. Thomas, the tutorial demonstrate Recoverable Interchange Processing in BizTalk Server 2004/2006/R2. Worth spending 9 minutes on the video.

To download the video please Click Here

Downloading the related sample files might also be helpful. They can be found here:
Click Here

Live Play (no need to download the file) of this video is available here:
Click Here

BizTalk Server R2 - WCF HTTP Binding through BizTalk WCF Adepter

2007-12-25T18:35:00.001+05:30

A very good video demonstration by Stephen W. Thomas, the video demonstrate WCF HTTP Binding through BizTalk WCF Adepter. Worth spending 13 minutes on the video.

To download the video please Click Here

Downloading the related sample files might also be helpful. They can be found here:
Click Here

Live Play (no need to download the file) of this video is available here:
Click Here

Presentation on .NET 3.0/3.5 at Microsoft, NET C# ASP.NET VB.NET WCF WPF WF LINQ

2007-12-24T01:40:00.000+05:30

Very good video presentation by instructor Andrew Troelsen delivers presentation on .NET 3.0/3.5 at Microsoft, NET C# ASP.NET VB.NET WCF WPF WF LINQ. Worth spending your busy time or office time ;-).

BizTalk Server 2006 - Assert Functoid - Training Video

2007-12-23T02:29:00.000+05:30

Another good video on YouTube, it demonstrates Assert Functoid.

http://www.youtube.com/watch?v=eypWfGlKSwM

BizTalk Server 2006 - Custom Mapping Functoid - Training Video

2007-12-23T02:24:00.000+05:30

A good Video available on YouTube network, it's very good demonstration for Custom Mapping Funcatoid. Enjoy pie!

http://www.youtube.com/watch?v=Q-CqDfPM7MU

WCF - Microsoft Service Configuration Editor – Services - SvcConfigEditor.exe

2007-12-21T02:47:00.001+05:30

Microsoft announced very recently about "Oslo" and their strategies about Enterprise Integrations and Service Orientation. The announcement made me tempt to explore WCF (bytheway I started calling me WCF professional few months ago ;-) ), and I come across to a very good tool called "Microsoft Service Configuration Editor". So grabbed the opportunity to document something and let's publish ;-) please bare one more article…

While learning WCF the first thing came to my mind was Enterprise Services Management (This days I am playing my cards with Live Service Management), it's ok if you plan 2-3-10-20-50 number of the services for your business but when you are planning services in big bulk, distributed deliveries of services, different assemblies, managing behaviors/contracts/endpoints for various clients/hosts, etc…. there could be N number of the possibilities for various service management scenarios. And my thoughts and excitement brought me to the administrative tool called "Microsoft Service Configuration Editor". Microsoft is supplying this tool with "Microsoft Windows SDK v6.0A" itself, it's very useful tool for the service management (especially WCF).

WCF is saving configuration settings in web.config / app.config files and it's very hard to manage manually by editing.

Below I am just trying to put some screens and my tutorial experience to you, hope you like it.

Here for the demonstration, I am using the sample scenario called "TrustedFacade " which you can find in Microsoft WCF Samples ("\WCF_WF_CardSpace_Samples\WCF\Scenario\TrustedFacade"). Not going inside the code or development part, I am going to focus only on the deployment and administration with the Configuration Editor.

Steps, for configuring the service:

You can open the tool from "Start Menu\Programs\Microsoft Windows SDK v6.0A\Tools\ Microsoft Service Configuration Editor" or go to the next path "Microsoft SDKs\Windows\v6.0A\Bin\SvcConfigEditor.exe".
Click on crate a new service, it will bring you one service creation wizard.
Click on the Browse Button and drive the wizard to the service dll, it can be GAC or File Location.
After selecting the service, click on Next button, it will bring you to service contract page. It will show you all contracts available in one dropdown list in service, an appropriate contract you select and then move ahead with the next phase of configuration.
Next screen will give you option to decide the communication mode, where you can select an appropriate mode of communication channels.
Next screen will bring you selection of interoperability option.
Now, the coming screen will ask you the address of your endpoint.
After performing the above steps, now your wizard is ready to create configuration.

Now your service has been configured, the tool also provide facility to manage other configuration like hosts, base addresses, multiple endpoints, binding configuration, contracts, listening uri, etc. I am here just focusing on the service configuration, later on I will publish client and other configuration options available with in the tool. So let's coming back to services, the below image shows you, how to add multiple base location in host.

You can also add multiple endpoints to the service, as well as you can add certifications, identity and headers to messages and channels. Please check the blow screens to understand the configurations options for end points.

After completion of configuration you can save it as *.config file, which need to manage with services at the IIS/service hosting location.

This tool can give very good flexibility with the management and administration of the WCF services on large scale.

I will back with some more interesting articles on WCF as my study progress with it.

Thanks to bear with me during the article, please send your feedback to nilayparikh@gmail.com.

Cheers!

Screencast: Microsoft BizTalk Server 2006 - A very good video

2007-12-20T21:43:00.000+05:30

Screencast: Microsoft BizTalk Server 2006. Cool video to understand BizTalk and scope of implementation. The video content requirement scenarios, basic development tool introduction, various artifacts, etc...

Source: Youtube

From: InfoWorld

Enjoy!

Now, we can chat live from Blog... :-) new cool chat tool

2007-12-20T20:47:00.000+05:30

Let's Talk,

New widget I have added to my Blog , please feel free to put some line there and suggestions are welcome.

Find this new widget at left pan of the page,

Thanks,

Hope you like this...

Understand BizTalk – Performance Counters – Bottleneck Analysis (Counter: Low % Disk Idle on SQL Server High Avg. Disk Queue Length on SQL Server)

2007-12-19T17:32:00.001+05:30

Counters: Low % Disk Idle on SQL Server High Avg. Disk Queue Length on SQL Server

Area of Analysis:
- Check whether the tracking database and Message box are on the same disks.
- Check whether the data and log files are on the same disks.
- Check the log sizes.

Possible Solutions:
- Use a SAN.
- Ensure that the tracking and Message-box databases are on different servers.
- If they are on the same server, ensure that they are on different disks.
- Ensure that the data and log files are not sharing the same disks.
Make sure the BizTalk agents are enabled on the server. The agents copy tracking data from the Message box to the tracking database. They also back up the databases and clean up the logs.

Understand BizTalk – Performance Counters – Bottleneck Analysis (Counter: Low % CPU Idle on SQL Server)

2007-12-19T17:30:00.000+05:30

Counter: Low % CPU Idle on SQL Server

Area of Analysis:
- Check if DBA changed any of the default database settings set by the BizTalk installation.
- Max Degree of Parallelism are set to off Auto-Update Statistics and Max Degree of Parallelism are set to off and 1 respectively on purpose.

Possible Solutions:
- Minimize the number persistence points in your orchestrations.
- Use static methods instead of wrapping nonserializable components in atomic scopes.
- Avoid using Parallel shapes, except when needed.
- In a multi-Message box scenario, ensure that you have at least three Message boxes. The master Message box is doing all the routing to secondary Message boxes, which is CPU intensive. Whenever you are using multiple Message boxes, Microsoft’s distributed Transaction Coordinator (DTC) is involved; therefore you need to jump from one Message box to three instead of only two to offset that overhead.

Understand BizTalk – Performance Counters – Bottleneck Analysis (Counter: - Low % CPU Idle on BizTalk Server)

2007-12-19T17:20:00.000+05:30

Counter: Low % CPU Idle (On BizTalk Server)

Area of Analysis:
- Too many hosts running on the server.
- Improper use of custom pipelines.
- Custom components requiring optimization.

Possible Solutions:
- Isolate receive, process, and send functionality into different hosts and run host instances on different servers within the BizTalk Server Group.
- Move message transformation out of orchestrations to your ports to avoid the creation of new copies.
- Move message filters to your ports and receive locations.
- Optimize your schema. Large schema reduce performance.
- Use distinguished fields in orchestrations rather than properties or XPath.
- Use pass-through pipelines whenever possible.

I will surely come back again on more performance & bottleneck analysis.

Exposing a simple BizTalk orchestration as WCF service

2007-12-19T02:27:00.000+05:30

Very good article about the ESB solution approach. It explains how you can expose the BizTalk orchestration in WCF Services!

Please visit below link for full article.

http://blogs.msdn.com/rgarg/archive/2007/11/19/exposing-biztalk-orchestration-as-wcf-service.aspx

Business Intelligence Demonstration - Case Study Video (SSAS/SSIS/SSRS)

2007-12-18T23:33:00.000+05:30

Great video and case study for Microsoft BI. I will recommend to spend your 27 minutes behind it, it's really worth.

Microsoft BizTalk Adapter for SAP – Part 1 (Introduction)

2007-12-18T22:16:00.001+05:30

The Microsoft BizTalk Adapter v2.0 for mySAP Business Suite i s a "no-code" connectivity solution that integrates SAP data seamlessly from SAP R/3 4.X and R/3 6.20 (Enterprise) systems with BizTalk Server. The adapter provides services to support the transport of IDocs, BAPIs, and RFCs (including customized IDocs) inbound and outbound between BizTalk Server and SAP.

The Adapter supports the following SAP schema types:

Business API (BAPI)
Intermediate Document (IDoc) versions 2.0 and 3.0
Remote Function Call (RFC)

Multiple IDoc, BAPI, and RFC schemas can be selected for each adapter in stance. The Adapter provides the following functionality:

Supports synchronous and asynchronous transactions
Provides full transactional support for all IDoc, BAPI, and RFC schemas
Converts SAP structures to XML Schema Definition language (XSD) format

Microsoft BizTalk Adapter for mySAP Business Suite includes:

Design-time (using the standard functionality available in the BizTalk adapter framework)
Schema generation wizard
BizTalk Explorer configuration pages
BizTalk Administrator configuration pages
Run-time
Transmitter -The transmitter sends IDocs, BAPIs, and RFCs to the appropriate SAP system via the .NET
Connector. If the SAP system is not available, the transmission can be retried, or sent to a suspend queue,
Depending on user configuration.
Receiver -The receiver acts as a listener, waiting for IDocs, BABIs, and RFCs to be sent by SAP via the .NET
Connector.
More than one receiver can be configured, in order to receive data from multiple SAP systems, and to
Provide load balancing.
Exception handling -The adapter provides full transactional support between SAP and BizTalk Server. If
There is a failure in receiving or sending, the transaction is rolled back.

Microsoft BizTalk adepter comes with two components, first is runtime module called BizTalk/SAP adepter, which is communicating with SAP. Another is design time component that integrates Visual Studio design time functionality to adepter.

This Wizard is use to connect to SAP through a port defined on machine, search lists of SAP functions (BAPIs and IDOCS), and select the function to call. The wizard generates schema (xsd) for messages you need to receive/ send from SAP. If the selected port is two way (Request/Response) then it generates two root schemas for the request and the response types.

BAPI/RFC Interface

The RFC Library offers an interface to a SAP System. The RFC Library is the most commonly used and installed component of existing SAP Software. This interface provides the opportunity to call any RFC function in a SAP System from an external application. Moreover, the RFC Library offers the possibility to write a RFC Server Program, which is accessible from any SAP System or external application. Most SAP Connectors use the RFC Library as communication platform to SAP Systems. SAP provides the RFC API in the form of C -routines, incorporated in the RFC Library. The RFC Library is available on all SAP Supported Platforms. The RFC Library is forward and backward compatible, i.e. an older release of the RFC Library can communicate with a SAP System at a higher version and vice versa.

IDoc Interface

IDoc, or Intermediate Document, is a standard SAP document format that allows to link different application systems via a message-based interface. The three main goals of IDocs are:

Structuring the exchange of business documents so they are automatically processed.
Reducing the structural complexity of different application systems to as simple a structure as possible.
Extensive exception handling before posting data to the application.
IDocs are defined and considered on two levels, the technical and the business level. The former allows them to support application-independent functions, e. g. routing and handling technical exceptions.

Referances:

MSDN Library
MS SAP Adepter Documentation

Very soon I will present the next part of the SAP Adepter Series.

Tools for testing BizTalk Application and Landscape

2007-12-18T18:11:00.000+05:30

Tools for testing BizTalk Application and Landscape

I have seen this links on the Darren Jefford's Blog. He is Principal Consultant at Microsoft UK. I am just putting consolidated list of resources across you. Enjoy the pie!

PerfMon Counter spreadsheet
http://media.wiley.com/product_ancillary/22/04700464/DOWNLOAD/Chapter9_Code.zipOrchestration
Profiler
http://www.codeplex.com/biztalkorcprofiler

BizTalk Documenter
http://www.codeplex.com/biztalkdocumenter

LoadGen 2007 (now includes WCF, Sharepoint 2007 support amongst others)http://www.microsoft.com/downloads/info.aspx?na=22&p=1&SrcDisplayLang=en&SrcCategoryId=&SrcFamilyId=&u=%2fdownloads%2fdetails.aspx%3fFamilyID%3dc8af583f-7044-48db-b7b9-969072df1689%26DisplayLang%3den

Old version of LoadGen 2004 (recommend you use the 2007 release moving forward)http://go.microsoft.com/fwlink/?LinkId=59841BizUnithttp://www.codeplex.com/bizunit

Codename "Oslo" - I am delighted

2007-12-18T18:07:00.000+05:30

Codename "Oslo"

Microsoft has just made the first public announcement of “Oslo” at the SOA and BPM conference, at a high level “Oslo” is an overarching initiative across multiple products and Microsoft divisions, in fact the first release as announced today will be made up of BizTalk Server “6”, BizTalk Services “1”, Visual Studio “10”, System Center “5” and .NET Framework “4.0”.

Overview
http://www.directionsonmicrosoft.com/samples/Olso1107.pdf

Oslo is driven out of Connected Systems Division (CSD) which is the almagamtion of the BizTalk, WCF, WF, ASMX, MSMQ, .NET Remoting, etc.

Sending Email using Biztalk orchestraion

2007-12-18T00:42:00.000+05:30

Sending Email using Biztalk orchestration

Very good video tutorial available on YouTube. I will recommend you to watch the Video to understand the play with Email or SMTP adepter in BizTalk.

Very good informative work Nikhil!

Please copy paste below link to address bar it will direct navigate you to the video location.

http://www.youtube.com/watch?v=BWngoSM2aQ8

Hope it helps you!

RFID Anywhere Integration with BizTalk

2007-12-18T00:37:00.000+05:30

RFID Anywhere Integration with BizTalk

A very good presentation on RFID implementation for BizTalk Server 2006.

BizTalk 2006 Pattern Wizard

2007-12-18T00:21:00.000+05:30

BizTalk 2006 Pattern Wizard

This presentation is on BizTalk 2006 Patterns Wizard which is mostly very unexplored part for many BizTalk professional, I would really recommend you to see the video and know the superb rapid development facility available to implement very complex integration patterns.

BizTalk 2006 TechNet Posters

2007-12-06T20:38:00.000+05:30

Some really good posters for BizTalk 2006 / R2 and other Microsoft technology
It's really interesting...

BizTalk Server 2006 Capabilities

http://www.microsoft.com/downloads/details.aspx?FamilyID=00a6472c-14df-4da8-8d77-6aa2315b5862&displaylang=en

BizTalk Server 2006 R2 Capabilities

http://www.microsoft.com/downloads/details.aspx?familyid=34F41573-C552-466F-B531-32CB09A57CDD&displaylang=en

BizTalk Server 2006 R2 Runtime Architecture

http://www.microsoft.com/downloads/details.aspx?familyid=8790E652-1DA5-4E80-88FE-B87606233DB4&displaylang=en

BizTalk Server 2006 R2 Scale-Out Configurations

http://www.microsoft.com/downloads/details.aspx?FamilyID=ff468298-64be-4947-a086-f61584caf995&displaylang=en

Furthermore:

Exchange

http://www.microsoft.com/downloads/details.aspx?FamilyId=FDCDF6E5-DE47-4B58-8086-282101BCDDE9&displaylang=en

Active Directory

http://www.microsoft.com/downloads/details.aspx?familyid=c236336d-ab43-44b1-ad6f-a2f668fb8c02&displaylang=en

Windows Server 2008 (Active Directory & Feature Components)

http://www.microsoft.com/downloads/details.aspx?FamilyID=c2b9e44e-0bbd-47cb-bc09-b3d48be7f867&DisplayLang=en\
http://download.microsoft.com/download/2/e/8/2e88c004-3b10-446f-9d5a-58af27deafc4/Windows%20Server%202008%20Active%20Directory%20Components.pdf
http://download.microsoft.com/download/2/e/8/2e88c004-3b10-446f-9d5a-58af27deafc4/Windows%20Server%202008%20Feature%20Components.pdf

.NET Framework 3.5 Common Namespaces and Types

http://www.microsoft.com/downloads/details.aspx?FamilyID=7B645F3A-6D22-4548-A0D8-C2A27E1917F8&displaylang=en
http://download.microsoft.com/download/4/a/3/4a3c7c55-84ab-4588-84a4-f96424a7d82d/NET35_Namespaces_Poster_LORES.pdf

2007 Office System Document: Developer

http://download.microsoft.com/download/0/9/c/09cda3f2-6d3d-4082-aec5-9a62b7679ecf/2007OfficeDeveloperMap.pdf
http://download.microsoft.com/download/0/9/c/09cda3f2-6d3d-4082-aec5-9a62b7679ecf/InfoPathObjectModelMap.pdf

Visual C# 2008 Keybinding Reference

http://www.microsoft.com/downloads/details.aspx?familyid=e5f902a8-5bb5-4cc6-907e-472809749973&displaylang=en

Visual Basic 2008 Keybinding Reference

http://www.microsoft.com/downloads/details.aspx?familyid=255b8cf1-f6bd-4b55-bb42-dd1a69315833&displaylang=en

Visual C++ 2008 Keybinding Reference

http://www.microsoft.com/downloads/details.aspx?FamilyID=4411BBFC-0E3C-42B3-BD05-AF1D292C986F&displaylang=en

Visual C++ 2005 Keyboard Shortcut Reference

http://www.microsoft.com/downloads/details.aspx?FamilyID=bccf84f4-4136-48b2-b4ec-83eaa484da20&DisplayLang=en

Visual Basic 2005 Keyboard Shortcut Reference

http://www.microsoft.com/downloads/details.aspx?familyid=6bb41456-9378-4746-b502-b4c5f7182203&displaylang=en&tm

Microsoft® Silverlight(tm) 1.1 Alpha Developer Reference

http://www.microsoft.com/downloads/details.aspx?FamilyID=811D8AD6-8D48-4684-B08C-686462D58A56&displaylang=en

PnP Overview

http://www.microsoft.com/downloads/details.aspx?FamilyID=d99fb3dd-e753-4500-bdad-e59ac0cea903&DisplayLang=en

Smart Client

http://www.microsoft.com/downloads/details.aspx?FamilyID=523bf8a4-2b48-459b-9a2a-19cfabad08dc&DisplayLang=en

Security Posters

http://www.microsoft.com/Education/SecurityPosters.mspx
http://download.microsoft.com/download/2/1/3/21343626-8172-4284-9d18-3a961bf71e00/SS_Blue_Posters.pdf

Introduction of Host Integration Server 2006

2007-11-11T01:33:00.000+05:30

Host Integration Server 2006

Microsoft® Host Integration Server 2006 provides comprehensive bidirectional services for integrating Microsoft Windows® with legacy systems. Host Integration Server 2006 extends Windows to other platforms by providing interoperability in three areas:

· Application Integration Services

· Data Integration Services

· Network Integration Services

Most of the services provided by Host Integration Server 2006 expose a programming interface, which enables you to extend the functionality of the product and integrate it more tightly in your own environment. This guide describes these interfaces and provides guidance on how to use them.

Overview

Microsoft Host Integration Server 2006 provides the key enabling technologies that allow customers to integrate their mission-critical host applications, data sources, messaging and security systems with new solutions developed using the Microsoft Windows® platform.

The following diagram shows an overview of the major features provided by Host Integration Server 2006.

One goal of Host Integration Server 2006 is to enable enterprise developers to move towards a services-oriented architecture (SOA) using the .NET Framework. Transaction Integrator (formerly COMTI) allows Windows developers to publish business rules in mainframe CICS and IMS applications as XML Web Services. The new TI Designer runs within Visual Studio® .NET, providing a more efficient developer experience. At the same time, TI has been extended to support AS/400 applications by offering an RPG Import Wizard and distributed program call runtime across a TCP/IP network connection. To directly publish vital data stored in IBM DB2, Host Integration Server 2006 offers a Microsoft Managed Provider for DB2 that is based on an improved Microsoft network client for DB2.

Host Integration Server 2006 includes innovative technologies, such as TI Host-Initiated Processing (HIP) that offers TCP/IP and SNA listeners that allow a Windows Server computer to function as a peer to IBM mainframe and AS/400 computers. For example, CICS programmers can call into Windows COM+ or .NET server components, just as if they were another CICS transaction program, by using familiar programming models such as CICS DPL (distributed program link). HIP allows enterprises to more effectively move portions of their application logic or data onto the more cost-effective Windows Server and SQL Server platforms, while ensuring logical and secure bi-directional application communication.

Finally, Host Integration Server 2006 offers industry-standard technologies allowing the network administrator to move towards pure TCP/IP WANs and datacenters. Transaction Integrator supports the CICS enhanced TCP/IP listener and persistent TCP/IP connections. The DB2 data providers support reliable 2-phase commit distributed transactions over TCP/IP to DB2 running on most popular computing platforms. The new IP-DLC Link Service supports the industry-standard SNA over IP routing so that branch-deployed and centrally-deployed Host Integration Server computers can connect directly to z900 mainframes via high-speed IP networks. No longer do enterprises have to remotely administer branch cluster controllers, utilize expensive data link switching (DLSw) capable routers, or maintain costly front end processors (FEPs). With Host Integration Server 2006, enterprises can continue to consolidate their network infrastructure, while supporting the same level of SNA-compatible applications and services.

Transaction Integrator

Transaction Integrator (TI) enables developers to integrate mainframe-based transaction programs (TPs) with component-based Microsoft® Windows® applications. With Transaction Integrator, you can integrate existing mainframe-based TPs with Windows-based COM or distributed COM (DCOM) applications. You may not have to modify your mainframe TP if the business logic is separate from the presentation logic. The wizards available in the TI Designer and TI Manager guide you through the process, step-by-step.

Transaction Integrator is appropriate when you need a synchronous or transactional solution where both systems being integrated are running at all times. For applications only requiring an asynchronous integration solution, a messaging-based solution using the MSMQ-MQSeries Bridge is preferred over Transaction Integrator.

Applications that integrate message queuing and that use MSMQ-MQSeries Bridge in a Host Integration Server 2006 environment can be developed using several different development tools and application programming interfaces including the following:

· C or C++ applications that use the MSMQ-MQSeries Bridge Extensions to extend the MSMQ-MQSeries Bridge.

· Microsoft Visual Basic® applications that use MSMQ-MQSeries Bridge Extensions to extend the MSMQ-MQSeries Bridge.

To use this guide effectively, you should be familiar with the following:

· Microsoft Host Integration Server 2006

· Microsoft Windows® 2000 or later

· Message Queuing

· IBM MQSeries

Transaction Integrator (TI) Designer

Transaction Integrator (TI) Designer is a graphical user interface for creating TI components, which are annotated type libraries or assemblies. It is a stand-alone program that does not need to be connected to the mainframe. You can use TI Designer to export or import the host data declarations used in mainframe Customer Information Control System (CICS) and Information Management System (IMS) programs or AS/400 programs.

TI Designer is hosted within the Microsoft® Visual Studio® .NET environment. You can use it to generate both Windows®-initiated processing (WIP) and host-initiated processing (HIP) objects. TI Designer also supports COM type libraries and .NET assemblies.

To install Transaction Integrator Designer

Transaction Integrator (TI) Designer is installed and configured by the Microsoft® Host Integration Server 2006 Installation Wizard. You can install and configure TI Designer at the time you first install Host Integration Server 2006, or you can use the Host Integration Server 2006 Installation Wizard at a later time to add TI Designer. The Host Integration Server 2006 Installation Wizard installs everything you need, including program files, Help files, the CedarBank sample application, and other tools. The TI Project template is automatically installed on your computer at :/Program Files/Microsoft Host Integration Server/System/Projects/.

Note TI Designer is hosted within the Visual Studio .NET development environment and must be registered in Visual Studio at the time of installation. Be sure that Visual Studio .NET is installed on your computer before you install TI Designer.

1. Launch Host Integration Server 2006 Setup and accept all defaults until you reach the Custom Installation page.

2. On the Custom Installation page, expand the Application Integration node.

3. Select Transaction Integrator Designer and any other options you want to install.

4. Click Next, and then follow the on-screen directions.

To create a TI Project

On the Start menu, To create a TI Project

1. On the Start menu, point to Programs, and then click Microsoft Visual Studio .NET.

2. On the Visual Studio .NET File menu, click New, and then click Project.

3. Under Project Types, select HIS Projects.

4. Under Templates, select TI Project.

5. After Name, type the name of the project. The name can be a maximum of 256 Unicode characters.

6. After Location, type or browse the location to store the project.

Introduction to Data Integration

You can develop applications for data integration used in a Host Integration Server 2006 environment using several different development tools and application programming interfaces including the following:

· C or C++ applications that use OLE DB to access AS/400 and VSAM files.

· C or C++ applications that use OLE DB to access IBM Data Base 2 (DB2).

· C, C++, or Microsoft® Visual Basic® applications that use Open Database Connectivity (ODBC) drivers to access IBM DB2.

· Visual Basic applications that use ActiveX® Data Objects (ADO) to access AS/400 and VSAM files.

· Visual Basic applications that use ADO to access IBM DB2 using OLE DB.

· Visual Basic applications that use ADO to access IBM DB2 using ODBC.

· C, C++, or Visual Basic applications that use the Host File Transfer ActiveX control to transfer files to and from MVS, OS/390, AS/400, and AS/36.

· C, C++, or Visual Basic applications that use the Data Queue ActiveX control to access AS/400 data queues.

· Microsoft C++ .NET, Microsoft C#, Microsoft Visual Basic .NET applications that use the Managed Provider for DB2 to access IBM Data Base 2 (DB2).

To use this section effectively, you should be familiar with:

· Microsoft Host Integration Server 2006

· One of the following operating environments:

· Microsoft Windows Server™ 2003

· Microsoft Windows XP

· Microsoft Windows® 2000 Server

· SNA concepts

Depending on the application programming interface and development tools used, you should be familiar with:

· Microsoft COM objects

· Microsoft OLE DB

· Microsoft ADO

· Microsoft ODBC

· Microsoft .NET

BEPL with BizTalk

2007-09-04T13:29:00.000+05:30

BEPL with BizTalk

Overview

BizTalk Server is part of Microsoft's Windows Server System family of products that enable you to integrate systems, employees, and trading partners. It is an integration server product (Middleware) used to connect different applications, systems, or business processes within an organization (EAI) or between organizations (B2B). It is also be used to aggregate services to build a Service-Oriented Architecture (SOA).

Content-Based Routing (CBR) where messages can be routed based on the values of certain fields inside the message
Support for BPEL4WS
Support for WSDL
Support for XSD
Support for XSLT
A Health and Activity Tracking (HAT) tool for tracking and debugging of Orchestrations and messages
A Business Rules Engine for abstracting business rules from the business process
Business Activity Monitoring for real-time monitoring of business process in the production environment
Business Activity Services including Trading Partner Management for integration with other business processes
Human Workflow Services for workflow systems involving human intervention, such as approvals, data entry, and collaborations

Support for BPEL and XLANG/s

BizTalk supports BPEL as a business process exchange format, to exchange the business process definition with other vendors. BizTalk does not natively execute BPEL, but can export business processes to BPEL and also import external BPEL files that define business processes.

In BizTalk, the business process is built visually using the Orchestration Designer inside Visual Studio and is internally stored and represented using XLANG/s.

XLANG/s is Microsoft's proprietary language for defining business processes in BizTalk Server 2004. It is an improved version of XLANG (used in earlier versions of BizTalk), with support for the .NET framework and for invoking .NET components. It is not mandatory for a developer to learn the underlying XLANG/s to build business processes as the Orchestration Designer automatically generates it.

While BPEL is platform neutral by design, XLANG/s is very specific to BizTalk and the .NET platform and as such is able to provide advanced functionalities and a broader range of services than BPEL. However, this also means that if you plan to export business processes to BPEL, you should carefully avoid using features specific to BizTalk and XLANG/s, such as message transformation using maps or calling .NET components from within an Orchestration.

BizTalk Orchestration Shape	BEPL Tag
Send shape	<invoke> or <reply>

Receive shape	<receive>
<receive>	Port
Port	<partnerLinks>, <partnerLink>
<partnerLinks>, <partnerLink>	Role Link
Role Link	<partnerLinkType>, <role>
<partnerLinkType>, <role>	Message Assignment shape
Message Assignment shape	<assign>, <copy>, <from>, <to>
<assign>, <copy>, <from>, <to>	Decide shape
Decide shape	<switch>, <case>, <otherwise>
<switch>, <case>, <otherwise>	Delay shape
Delay shape	<wait>
<wait>	Listen shape
Listen shape	<pick>, <onMessage>, <onAlarm>
<pick>, <onMessage>, <onAlarm>	Parallel Actions shape
Parallel Actions shape	<flow>
<flow>	Loop shape
Loop shape	<while>
<while>	Scope shape
Scope shape	<scope>
<scope>	Throw Exception shape
Throw Exception shape	<throw>
<throw>	Compensate shape
Compensate shape	<compensate>
<compensate>	Suspend shape
Suspend shape	Not supported in BPEL
Not supported in BPEL	Call Orchestration shape
Call Orchestration shape	Not supported in BPEL
Not supported in BPEL	0
	Start Orchestration shape
Start Orchestration shape	Not supported in BPEL
Not supported in BPEL	Call Rules shape
Call Rules shape	Not supported in BPEL
Not supported in BPEL	Transform shape
Transform shape	Not supported in BPEL
Not supported in BPEL	Terminate shape
Terminate shape	<terminate>
<terminate>	Compensation Block
Compensation Block	<compensationHandler>
<compensationHandler>	Exception Handler
Exception Handler	<faultHandler>, <catch>, <catchAll>
<faultHandler>, <catch>, <catchAll>	Correlation
Correlation	<correlation>, <correlations>, <correlationSets>, <correlationSet>

Exporting Orchestration to BPEL

In the last section we saw how to build an Orchestration in BizTalk. In this section we will export the same to BPEL.

To export the Orchestration to BPEL, we have to do the following:
Keeping the Orchestration Designer selected in the foreground, bring up the properties window such that it displays the Orchestration properties.
You will see that the Module Exportable property is set to False. Setting this property to true will make two other properties visible in the properties window (Module XML Target Namespace and Orchestration Exportable).
Set a value for Module XML Target Namespace such as http://packtpub.com/bpel/example and set the Orchestration Exportable to True. This will make another property visible, called the Orchestration XML Target Namespace, shown in the figure below.
Set a value for Orchestration XML Target Namespace and you are all set to export the Orchestration to BPEL.
In the Solution Explorer window right-click on the Orchestration file (InsuranceSelectionProcess.odx) and select Export to BPEL.

Importing BPEL Processes into BizTalk

Importing BPEL processes into BizTalk is quite easy. BizTalk provides a separate project template called the BizTalk Server BPEL Import Project, which will bring up a wizard to take you through the BPEL import procedure step by step. However, there are a few things you need to take care of before importing business processes:

Ensure that the Name property of the WSDL definition node and that of the BPEL process node are not the same.
Do not use any XLANG/s reserved words in your BPEL.
Use only simple types predefined in XSD.

To import the .bpel and the .wsdl files into BizTalk. Here are the step-by-step instructions:

Open a new BizTalk Server BPEL Import Project in Visual Studio .NET and name it AsyncTravelProcessBPELImport as shown overleaf.

Click OK to bring up the BPEL Import Wizard shown below.

Click Next, and then Finish.

After importing the travel process example, you can see in the Solution Explorer of Visual Studio that the wizard has created four orchestration (.odx) files and two schema (.xsd) files shown below:

The wizard breaks down each .wsdl file into a schema file (.xsd) that contains the schema definition and an orchestration file (.odx) that contains the other things like Port Types, Message Types, and Partner Link Types (known as Role Link Types in BizTalk). This orchestration file does not contain any business process logic and is only used to define the types.

The .bpel file is converted into an orchestration file (.odx) that contains the actual business process logic and declarations for things like Partner Links (Ports and Role Links in BizTalk) and Variables (Messages in BizTalk).

Comparing BizTalk Orchestration Constructs with BPEL

Receive and Send Shapes (<receive>, <invoke>, <reply>)

Messages are received and sent to the ports (partnerLinks) within the BizTalk Orchestration using the Receive and Send shapes. These are equivalent to the <receive>, <reply>, or <invoke> keywords of BPEL. Note that the same Send shape is used to send messages for both One-Way (Asynchronous <invoke>) and Request-Response (Synchronous <reply>) calls.

Port and Role Link Shapes (<partnerLink>, partnerLinkType>, <role>)

Ports and Role Links are the endpoints for messages entering or exiting the Orchestration. They are similar to partner links in BPEL. Configuring the ports includes setting the PortType, Communication Direction (send or receive), Communication Pattern (One-Way or Request-Response), Delivery Notification, and Binding to the physical ports, among other things. When you drop a Port shape or a Role Link shape onto the side panels of the Orchestration Designer surface, the Port Configuration Wizard pops up and assists you in configuring the ports. Role Link is used to group together multiple PortTypes, similar to <partnerLinkType>.

Expression and Message Assignment Shapes (<assign>, <copy>, <from>, <to>)

Expression and Message Assignment shapes, as the name suggests, are used to implement business logic that contains expressions, loops, conditions, and assignments. It is possible to use XPath and XLANG/s within these shapes and also to call any external .NET components by defining and instantiating a variable of the particular .NET type.

Decide Shape (<switch>, <case>, <otherwise>)

A Decide shape is used for conditional branching. It is similar to an IF/THEN/ELSE/ELSIF programming construct or the <switch>, <case>, and <otherwise> constructs of BPEL. It will always have a minimum of two branches. Each branch (other than the Else branch) will have a Rule shape containing the condition expression to be evaluated. The condition expression of each branch is evaluated and if it satisfies the condition, that branch is executed; otherwise it will evaluate the condition of the next branch (if available) and so on. If none of them satisfy the condition the else branch is executed.

Delay Shape (<wait>)

The Delay shape is equivalent to the <wait> activity of BPEL. It is used to delay the execution of the process for a specific length of time (specified by the System.TimeSpan class) or until a particular time or date is reached (specified by the 310 System.DateTime class).

Parallel Actions Shape (<flow>)

The Parallel Actions shape is used to simultaneously execute multiple actions. It is similar to the <flow> activity of BPEL. It is possible to have many parallel branches and the process will wait till all the branches have completed before proceeding further.

Loop Shape (<while>)

The Loop shape is used to iterate through a set of actions as long as the Boolean condition evaluates to True. This is equivalent to the <while> activity of BPEL.

Suspend Shape

A Suspend shape is used to stop a running Orchestration and save its state such that it is possible for the administrator to resume the Orchestration instance. You use the Suspend shape if your Orchestration reaches a condition that is beyond the capabilities of the Orchestration, such that you would rather allow an administrator to look into it and resume the operations based on the administrator's decision. When an Orchestration is suspended an error is logged and it is possible to specify a string containing a brief description along with the error message to help the administrator diagnose the problem. This activity is not available in BPEL.

Terminate Shape (<terminate>)

A Terminate shape is used to abruptly stop all activities of a running Orchestration. This is equivalent to the <terminate> activity of BPEL. When an Orchestration is terminated an error is logged and it is possible to specify a string containing a brief description along with the error message to help the administrator diagnose the problem.

Integration with Web Services

BizTalk Server 2004 has native support for web services. It has a built-in SOAP adapter to directly communicate with web services. An Orchestration can consume web services by adding a web reference of the external services' provider from within the Visual Studio .NET environment. It is also possible to easily expose business processes as web services using a tool called Web Services Publishing Wizard, so that they can be used by external systems.

I recommend the book "Business Processes Execution Language for Web Services" to learn more exciting futures of BizTalk and it's compatibility with BPEL.

Bibliography

Books:

Business Processes Execution Language for Web Services

Web References:

http://www.oasis-open.org/committees/download.php/2046/BPEL%20V1-1%20May%205%202003%20Final.pdf

http://dev2dev.bea.com/webservices/BPEL4WS.html

http://www.ibm.com/developerworks/library/specification/ws-bpel/

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/dnbizspec/html/bpel1-1.asp

Persistence points in BizTalk

2007-05-08T09:34:00.000+05:30

Persistence points in BizTalk

The orchestration engine saves the state of a running orchestration instance at various points. If the engine must rehydrate the orchestration instance, start from a controlled shutdown, or recover from an unexpected shutdown, the engine runs the orchestration instance from the last persistence point, as if nothing else had occurred. For example, if a message is received and if an unexpected shutdown occurs before the state can be saved, the engine does not record that it has received the message. Instead, the engines receive the message again after it restarts. The engine saves the state of an orchestration in the following circumstances:

· The end of a transactional scope is reached.

· The engine saves the state at the end of a transactional scope. Therefore, the point at which the orchestration resumes is defined unambiguously. Compensation can be performed correctly, if it is necessary.

· The orchestration continues to run from the end of the scope if persistence was successful. Otherwise, the appropriate exception handler is invoked.
If the scope is transactional and atomic, the engine saves the state at the end of the atomic scope when the scope commits.

· If the scope is transactional and long-running, the engine generates a new transaction and persists the complete state of the runtime when the scope is completed.

· A debugging breakpoint is reached.

· A message is sent. The only exception to this is when a message is sent from an atomic transaction scope.

· The orchestration starts another orchestration asynchronously, as it does with the Start Orchestration shape.

· The orchestration instance is suspended.

· When the orchestration engine is asked to shut down, the orchestration engine tries to save control information and the current state of all running orchestration instances. This behavior lets the orchestration engine resume running orchestration instances when the engine is started again. If the orchestration engine cannot save the current state, the orchestration engine resumes the orchestration instance from the last persistence point that occurred before the shutdown. This behavior applies to the system shutdown in a controlled condition and after an abnormal termination.

· The engine determines that the instance should be dehydrated.

· The orchestration instance is finished.

Know BizTalk

2007-03-15T02:25:00.000+05:30

Know BizTalk...

BizTalk is an industry initiative headed by Microsoft to promote Extensible Markup Language (XML) as the common data exchange language for e-commerce and application integration on the Internet. While not a standards body per se, the group is fostering a common XML message-passing architecture to tie systems together. BizTalk says that the growth of e-commerce requires businesses using different computer technologies to have a means to share data. Accepting XML as a platform-neutral way to represent data transmitted between computers, the BizTalk group provides guidelines, referred to as the BizTalk Framework, for how to publish schema (standard data structures) in XML and how to use XML messages to integrate software programs.

What is BizTalk?

BizTalk is a business process management (BPM) server that enables companies to automate and optimize business processes. This includes powerful, familiar tools to design, develop, deploy, and manage those processes.
Everything you need for easy installation is included—integrated management tools, support for Web services, and a new business activity monitoring (BAM) portal. In addition, BizTalk Server 2006 includes 23 application and technology adapters in the box for connecting to legacy systems (mainframe and mid-range) and line-of-business applications (SAP, Siebel, PeopleSoft, Oracle, and JD Edwards).

Q.What is BizTalk Server?
A. BizTalk is a Business Process Management Server designed to connect the people, processes and information that you need to effectively manage and grow your business. By connecting applications within and between organizations, and providing the tools to let business staff define, create and modify the business processes that depend on that software, BizTalk Server provides real-time visibility into critical business functioning, maximizing your organizational agility. BizTalk Server 2006 is the fourth major release of the product, and offers dependable, scalable and enterprise-ready business process management.

Q.What does Business Process Management (BPM) mean?
A. Business process management refers to the automation of manual processes and the subsequent optimization of those processes. The goal of management is to improve the flow of business processes such that efficiencies are improved, risks are lessened and costs are lower.Components of Microsoft’s Business Process Management Solutions Business Process Management Tools and technologies to manage business processes that span applications and organizational boundaries.Application Integration Internal application integration within an enterprise using either standard or proprietary communication protocolsBusiness-to-Business Integration External integration with trading partners, customers or suppliers using either standard or proprietary communication protocols

Q.What is Microsoft’s business process management strategy?
A. Early business process automation approaches were comprised of many independent technologies that separately met needs for business process management and integration, but together were expensive and hard to manage. Microsoft’s strategy has been to align these technologies, enabling organizations to more readily automate and optimize processes, thereby improving efficiencies, decreasing costs and improving business value.

Q.Our company isn’t a large enterprise. Why would it be important for us to automate and manage our businesses processes?
A. More and more large customers and suppliers are demanding that the businesses in their supply chain automate in order to eliminate inefficiencies and reduce costs. While these mandates and government regulations impose a burden on small and midsize companies, achieving compliance (with Sarbanes-Oxley Act, for example) promotes business process transparency and better process controls, both of which can greatly benefit the organization. If a business analyst needs to exchange information with a trading partner, or needs to obtain information that resides within the company but on different computer systems, applications, or in different parts of the organization, then business process automation and management—both between and within businesses—can provide the solution.

Q.What business advantages can BizTalk Server offer midsize companies?
A. BizTalk Server provides a single solution for business process management, enabling companies to effectively communicate both within organizations and between trading partners. BizTalk Server not only connects legacy systems and applications together, it also lets you define your organizational business processes, connect the process to the people and systems, and manage the events and key performance indicators within each process in order to ensure that the business is running effectively and efficiently.

Q.What IT benefits can we realize from installing BizTalk Server?
A. Automating and managing your connections with trading partner applications and applications within your organization is a time consuming process, and BizTalk Server’s reusable integration services speeds deployment time, helping reduce IT administrative effort. Once these connections are made, BizTalk Server provides a single tool to create business processes and monitor them in real time, as well as to extract business intelligence, simplifying both IT workloads and reducing management overhead. And because BizTalk Server uses standards like XML and web services, IT administration is less complex.

Q.Why should we choose a Microsoft business process management solution?
A. Microsoft’s BizTalk Server offers great economics in business process management technologies, enabling midsize companies to install, configure and customize their BPM solution easily and affordably. Developers who are familiar with Microsoft tools will quickly become productive. And by using a common set of reusable services, organizations can reduce the costs and time to value of business process automation, while at the same time simplifying management and maintenance costs. In addition, by using BizTalk Server to provide BPM within your organization and between your trading partners, you also benefit from the long list of partners that have helped to make Microsoft’s solution one of the most versatile in the industry. Thousands of systems and devices are supported on the Microsoft platform, as well as over 2500 applications. Microsoft’s strong relationships with leading global and regional systems integrators has allowed us to deliver tailored solutions for every industry in every geography.

Q.We are looking for a business process management solution that our key business personnel can readily use. Does BizTalk Server provide easy-to-obtain and readily useable business intelligence?
A. BizTalk Server makes it easy to bridge the gap between what business users need to know and the underlying technology that IT staff orchestrate to make the software give you what you need. BizTalk Server allows you to map out business processes in a graphical format, making it straightforward for business personnel to understand, analyze and adjust business processes throughout the organization. Behind the scenes, IT staff map the graphical representation of each business process into simple XML schemas that make the integration process work. Once the business processes are set up, users can readily monitor business activities to obtain real-time information about the status of any business process, as well as receive notifications about any event or milestones of interest. Such output can easily be integrated with SharePoint and Excel (using a wizard), to provide graphical data output as well.

Q.What is the difference between accelerators and adapters, and how do we know if our company needs them?
A. BizTalk adapters extend the functionality of BizTalk Server, enabling it to connect to other servers, applications and Web services. Specifically, they remove the need to develop infrastructure components such as network protocol support and translation, data conversion, data transformation, in the development of business process automation solutions. BizTalk Server adapters are "no-code" connectivity solutions, and are provided in-box with BizTalk Server 2006.BizTalk accelerators provide developers with tools, data schemas, and processes to help significantly reduce the time required to develop a custom business solution in specific industries, such as health care, financial services, and supply-chain management. If you are required to comply with specific industry standards, you will likely require an accelerator.

Q.We are doing a lot of work with external partners, and we want a solution that will enable us to manage access to our corporate information simply but securely. How does BizTalk Server handle this?
A. Like the Windows Server operating system, BizTalk Server is more secure by design—a critical attribute, since the server acts as a gateway through which to send and receive documents within and between organizations. That said, the security requirements of your BizTalk Server deployment are highly dependent on factors unique to your organization, including your particular business needs, partners, company size, and so on. Every application for BizTalk Server is likely to have its own unique set of security requirements; and performance, availability, and cost considerations further influence these requirements. BizTalk Server 2006 provides robust support for securing data, such as encryption and digital signatures. Additionally, BizTalk Server 2006 offers single sign-on for users requiring authentication across multiple systems.

Q.Where will Microsoft go next with BizTalk Server?
A. BizTalk Server was one of the first products in the space to make a big bet on XML and Web Services. We will continue to embrace emerging technologies that help customers decrease the cost associated with automating and managing business processes. We will remain very focused on building software that delivers our vision of revolutionizing the manner in which organizations conduct business by coordinating the activities of their business applications, their people and their partners.

Q.A. Integration with Microsoft OfficeQ.How does BizTalk Server work with Office System products like SharePoint and InfoPath?
A. In combination with InfoPath, developers can design structured workflows including form-based information. BizTalk Server has native support for InfoPath enabling users to:• Submit InfoPath documents to BizTalk Server through web services• Submit and receive InfoPath documents through SharePoint Form Libraries• Manipulate the InfoPath form contents to create new forms• Update information in an existing form within BizTalk Server.With BizTalk Server 2006, customers can tap into SharePoint Portal Server to share BAM reports, manage trading partner relationships, and enhance human-based workflow capabilities within specific business process automation scenarios

Q.A. Integration with Microsoft DynamicsQ.How does BizTalk Server Integrate with Microsoft Dynamics ERP solutions?
A. Microsoft Dynamics, the next generation enterprise resource planning, brings the ERP suite of products onto the .Net platform, along with BizTalk Server, and brings roles-based user interfaces to the products that BizTalk integrates with, including Office, SharePoint and others.

Q.A. Integration with Visual StudioQ.I’ve heard of using Visual Studio for application integration. Does that undercut BizTalk Server?
A. Visual Studio and BizTalk Server are very complimentary technologies. Visual Studio is the premiere development environment for building XML Web Services and BizTalk Server 2006 is the premiere development environment for orchestrating Web Services into business processes. With BizTalk Server 2006, customers will find tight integration with Visual Studio, giving developers a familiar, unified development environment. The combination of Visual Studio and BizTalk Server will give developers a significant advantage as they build application integration and business process management solutions.

Q.A. Product Release InformationQ.What advantages does BizTalk Server 2006 offer over the 2004 release?
A. BizTalk 2006 offers a simplified installation and configuration process, and offers the IT administrator a more productive management and deployment process. Additionally, business user capabilities are extended, providing real-time support for business activities and alerts.

Q.How easy is it to upgrade from BizTalk Server 2004 to the latest release?
A. The upgrade from BizTalk 2004 to 2006 is seamless, reducing the time and costs associated with upgrading to new technologies.

Q.What platforms does BizTalk Server support?
A. Windows 2000, Windows XP Professional, Windows Server 2003, SQL Server 2000, SQL Server 2005, Visual Studio 2005, .NET Framework 2.0. In addition, BizTalk Server offers 64-bit run time support. BizTalk Server also supports interoperability with non-Microsoft platforms, and with its robust adapter framework, BizTalk can quickly connect to proprietary systems.

Q.What Microsoft products does BizTalk Server provide integration for?
A. BizTalk Server integrates directly with SQL Server, Visual Studio, InfoPath, SharePoint and Microsoft Office. In addition, Microsoft is committed to providing adapters for the most commonly used line-of-business applications and technologies.

Q.Our company is going through a grow phase. Will BizTalk Server be able to grow with us?
A. BizTalk Server offers enterprise level scalability and performance. The 2006 Enterprise edition offers x64 capabilities, and when paired with Intel or AMD-based servers, solutions can easily be scaled up—in low cost increments—over time. The Enterprise edition also provides simple hardware scale-out capabilities and has built-in load balancing.

Q.What services and support do you offer for BizTalk Server?
A. Microsoft offers a range of support options, whether you require on-site or phone-based support. New to Microsoft are technology centers which provide the facility and resources to determine whether the BizTalk integration solution will work for you. For ongoing information, join the BizTalk Community to share great new ideas. For more information on support and services, go to BizTalk Support.

Q.Our organizations require solutions that support load balancing and clustering. Does BizTalk support these solutions?
A. BizTalk Server 2006 Enterprise Edition provides built-in support for load balancing, helping organizations to deliver 24/7 availability for their mission critical solutions across their enterprise. Additionally, BizTalk is engineered to be fault-tolerant, and employs easily-configured and manageable SQL Server clustering to ensure no single point of failure.

Exception handling With Ports in BizTalk

2006-12-26T16:10:00.000+05:30

Exception handling With Static and Dynamic Ports in BizTalk ServerException handling With Static and Dynamic Ports in BizTalk Server

Introduction

In one of the interface that we developed for our client had following requirement.

Send the output file to target location using Dynamic send port. If Output file is already present at the target location then send the file to Backup location using a Static send port. If file is already present at backup location then catch the exception and send the file to error location using Static send port.

Technical Implementation and Usage

We’ll cover the technical implementation in following sections, with each subsequent section (optionally) making use of the previous section’s implementation.

Schemas, MAP and Pipelines

In order to implement this solution we have used following schema ,Map and Pipelines

1) schema : EmpData.xsd

2) Map :EmpTO Emp.btm

3) Receive Pipeline1

4) Send Pipeline1

Create one receive and three send ports in the orchestration in the orchestration view.

Here we will cover the creation of Dynamic send port.

2.2 Creating Dynamic Port and Exception Handling

Please see the below images for creating Dynamic send port

1) Create a port type in orchestration view (eg.PortType_DyanamicSend)

2) Add new port in orchestration view and configure the Port using Configuration Wizard

3) Drag a expression shape from the tool bar write following code in the expression shape

{

DynamicSend(Microsoft.XLANGs.BaseTypes.Address) = @"file://E:TempOUT" + strFilename;

}

This will create the dynamic port for you after deploying the orchestration.

(Note: Please note that we can read the transport type and Location from Configuration Section. Here I have hard coded transport type and location in the orchestration for simplicity)

To handle the exception, Set the Property Delivery Notification = Transmitted for each port you have created in orchestration view.

The exception object we need to construct is of type Microsoft.XLANGs.BaseTypes.DeliveryFailureException.

Now deploy the solution containing schemas, pipeline , orchestration.

Perform following steps:

 Create one Receive port in BTS explorer (e.g. DynamicPortExp.Receive with one receive Location

DynamicPortExp.Receive.File with File as transport type and e:dataIn as file drop location)

 Create two send ports in BTS explorer (e.g. DynamicportExp.Send.File with backup location as e:databackup and DynamicPortExp.Send.Error.file with error location e:dataError with transport type as File)

 Now drop the file (e.g. Emp1.txt) at Input location after processing through orchestration it reaches to a location specified by dynamic send port E.g. E:TempOUT.

 Once again drop the same file at input location as the file is already present at location specified by dynamic send port it will throw a Delivery failure exception we are catching this exception in the catch block of the orchestration. After logging this exception in the event viewer the file is sent to backup location.

 Again drop the same file at input location. After processing orchestration try to send the file at location specified by dynamic port as file is already present at the location it raises exception and try to send the file at backup location but file is also present at backup location. So we are expecting that orchestration will send this file to an error location but it will not send the file to error location and we will see the following warning message in the event viewer and orchestration will not catch the exception.

6) In order to catch the exception for static send port the trick here is to set the retry count(default –3) and retry interval(default –5) property of send port to Zero(0) as shown below