Monday 28 November 2011

WCF Proxy External to Internal and Avoid Serialization Overheads


In the past on numerous projects I had to consider application security as part of my overall software design and implementation. All internal systems that are required for the service are located in Internal Network layer protected by a firewall. The external world would not have direct access to the internal systems except via proxy from an designated external server to internal server.
The diagram below is a typical example of what this entails.



The External servers would only have access to the designated Internal Servers and from there the internal servers would do the processing and communicating to backend systems before responding back via the external systems.
The most efficient way I found to do this is to effect create a proxy External WCF service that would then forward that request to the internal systems without any serialization.
Essentially the method here being demonstrated is GetPrices.
Externally it will be exposed as
Message GetPrices(Message value);
Internally it will be the deserialized form of
GetPricesResponse GetPrices(GetPricesRequest request)

The conversion is done automatically by wcf. There are a few gotchas so please read the implementation especially around the message http headers. This is specific for wcf – rest services.
Let me demonstrate
First lets create an Interface called IGetPrices. It will be a REST WCF Service for this demonstration. It will have a method called GetPrices.
    [ServiceContract]
    public interface IGetPrices
    {
        [OperationContract(Action = "*", ReplyAction = "*")]
        [WebInvoke(Method = "POST", UriTemplate="", BodyStyle = WebMessageBodyStyle.Bare, RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json)]
        Message GetPrices(Message value);
    }

Then the implementation will be as follows

        public Message GetPrices(Message msgRequest)
        {
     //THE EXTERNAL WCF SERVICE WILL CREATE NOW ACT AS A CLIENT TO CALL INTERNAL GETPRICES     
            InternalGetPricesClient igpcGetPrices = new InternalGetPricesClient();
            HttpRequestMessageProperty reqProperty = (HttpRequestMessageProperty)msgRequest.Properties[HttpRequestMessageProperty.Name];

            string strHeaderValue = "";
            for (int ihKey = 0; ihKey < reqProperty.Headers.Count; ihKey++)
            {
                strHeaderValue += reqProperty.Headers[ihKey];
            }
            System.Diagnostics.Debug.WriteLine("[GetPrices].External.Headers", strHeaderValue);

            //SOMETIMES EXTRA HEADERS CAN CAUSE AN ISSUE
            int ihFoundKey = 0;
            while (ihFoundKey < reqProperty.Headers.Count)
            {
                if ((reqProperty.Headers.GetKey(ihFoundKey).ToUpper() == "CONTENT-LENGTH") || (reqProperty.Headers.GetKey(ihFoundKey).ToUpper() == "CONTENT-TYPE"))
                {
                    ihFoundKey++;
                }
                else
                {
                    reqProperty.Headers.Remove(reqProperty.Headers.GetKey(ihFoundKey));
                    ihFoundKey = 0;
                }
            }

            //PASS CUSTOMISED HEADER TO INTERNAL CLIENT
            //WebOperationContext.Current.IncomingRequest.Headers.Add("AppFabricID", appFabric.ActivityID);

            msgRequest.Headers.To = igpcGetPrices.Endpoint.Address.Uri;
            msgRequest.Properties.Via = igpcGetPrices.Endpoint.Address.Uri;

            Message rspResponse = igpcGetPrices.GetPrices(msgRequest);

            //CAN CHECK RESPONSE HEADER TO SEE IF THERE IS AN ERROR
            HttpResponseMessageProperty respProperty = (HttpResponseMessageProperty)rspResponse.Properties[HttpResponseMessageProperty.Name];
            //if (respProperty.Headers["ERROR"] != null)
            //{
            //    throw new Exception(respProperty.Headers["ERROR"], HttpStatusCode.InternalServerError);
            //}
            rspResponse.Properties[HttpResponseMessageProperty.Name] = null;

            return rspResponse;
        }

Then the implementation for the external to internal client for InternalGetPricesClient igpcGetPrices = new InternalGetPricesClient(); will be.

    public partial class InternalGetPricesClient :  System.ServiceModel.ClientBase<IGetPrices>, IGetPrices
    {
        public InternalGetPricesClient()
        {
        }
        public InternalGetPricesClient(string endpointConfigurationName) :
            base(endpointConfigurationName)
        {
        }

        public InternalGetPricesClient(string endpointConfigurationName, string remoteAddress) :
            base(endpointConfigurationName, remoteAddress)
        {
        }

        public InternalGetPricesClient(string endpointConfigurationName, System.ServiceModel.EndpointAddress remoteAddress) :
            base(endpointConfigurationName, remoteAddress)
        {
        }

        public InternalGetPricesClient(System.ServiceModel.Channels.Binding binding, System.ServiceModel.EndpointAddress remoteAddress) :
            base(binding, remoteAddress)
        {
        }

        public Message GetPrices(Message request)
        {
            return base.Channel.GetPrices(request);
        }
    }


ALSO 

The Web.Config needs to be configured accordingly to configure the client accordingly.

<?xml version="1.0" encoding="utf-8"?>
<system.serviceModel>
       <serviceHostingEnvironment aspNetCompatibilityEnabled="true" />
       <bindings>
              <basicHttpBinding>
                     <binding name="soapBinding">
                           <security mode="None">
                           </security>
                     </binding>
              </basicHttpBinding>
              <webHttpBinding>
                     <binding name="webBinding">
                     </binding>
              </webHttpBinding>
       </bindings>
       <client>
              <endpoint name="Centrebet.PYOF.External.WCF.Interfaces.ExternalGetPricesClient"
                                  address="http://localhost:24984/GetPrices"
                                  binding="webHttpBinding"
                                  bindingConfiguration="webBinding"
                                  behaviorConfiguration="poxBehavior"
                                  contract="Centrebet.PYOF.External.Interfaces.IGetPrices"/>
       </client>
       <behaviors>
              <endpointBehaviors>
                     <!-- plain old XML -->
                     <behavior name="poxBehavior">
                           <webHttp/>
                     </behavior>
                     <!-- JSON -->
                     <behavior name="jsonBehavior">
                           <enableWebScript/>
                     </behavior>
                     <!-- <behavior name="ProtoBufSerializationBehavior">
                                  <ProtoBufSerialization/>
                                  </behavior> -->
              </endpointBehaviors>
              <serviceBehaviors>
                     <behavior>
                           <serviceThrottling maxConcurrentCalls="200" maxConcurrentSessions="200" maxConcurrentInstances="200"/>
                           <!-- To avoid disclosing metadata information, set the value below to false and remove the metadata endpoint above before deployment -->
                           <serviceMetadata httpGetEnabled="true"/>
                           <!-- To receive exception details in faults for debugging purposes, set the value below to true.  Set to false before deployment to avoid disclosing exception information -->
                           <serviceDebug includeExceptionDetailInFaults="false"/>
                     </behavior>
              </serviceBehaviors>
       </behaviors>
</system.serviceModel>

Now for the Internal Server
It will now deserialize the following way
namespace Internal.WCF.Interfaces
{

    [ServiceBehavior(ConcurrencyMode=ConcurrencyMode.Multiple, InstanceContextMode=InstanceContextMode.PerCall)]
    [ServiceErrorBehavior(typeof(GetPricesErrorHandler))]
    [AspNetCompatibilityRequirements(RequirementsMode = AspNetCompatibilityRequirementsMode.Allowed)]
    public class GetPricesService : IGetPrices
    {
        public GetPricesResponse GetPrices(GetPricesRequest request)
        {
            GetPricesResponse gprResponse = new GetPricesResponse();
            return gprResponse;
        }    
    }
}

Here is the implementation class.
namespace Internal.Interfaces
{
    // NOTE: You can use the "Rename" command on the "Refactor" menu to change the interface name "IService1" in both code and config file together.
    [ServiceContract]
    public interface IGetPrices
    {
        [OperationContract(Action = "*", ReplyAction = "*")]
        [WebInvoke(Method = "POST", UriTemplate = "", BodyStyle = WebMessageBodyStyle.Bare, RequestFormat = WebMessageFormat.Json, ResponseFormat = WebMessageFormat.Json)]
        GetPricesResponse GetPrices(GetPricesRequest request);
    }
}

There you have it how to create a WCF Proxy External to Internal and Avoid Serialization Overheads


AppFabric Event Logger Custom SQL Bulk Copy


I recently needed to customize the AppFabric logger to write to a different location apart from a SQL Database. Sadly to say out of the box it only comes with one provider. When configuring AppFabric in IIS Microsoft documentation states that you can provide your own providers.

Sadly to say I was not able to have 2 providers side by side and choose one from the other. Also writing a provider is not a simple task. So what I choose to do was implement my own IBulkCopy implementation. Essentially the three properties and WriteToServer completed the implementation needed for IBulk Copy.

To register my own AppFabricBukCopyProvider I essentially edited the Machine.Config as provided my own implementation dll.

Note: The lockElements of the machine.config forces the subsequent application app.config or web.config not to use bulkCopyProviders so I had little choice but to edit the machine.config.

Here is a sample

Edit Machine.Config

  <microsoft.applicationServer>
    <monitoring lockElements="bulkCopyProviders, collectors">
      <bulkCopyProviders>
        <!-— ORIGINAL ITEM <bulkCopyProvider providerName="System.Data.SqlClient" type="Microsoft.ApplicationServer.Monitoring.EventCollector.SqlServerBulkCopy, Microsoft.ApplicationServer.Monitoring, Version=1.0.0.0, Culture=neutral, PublicKeyToken=31bf3856ad364e35"/> -->
       <!-- Your own implementation -->
      <bulkCopyProvider providerName="System.Data.SqlClient" type="Helper.Provider.AppFabricBulkCopyProvider, Helper, Version=1.0.0.0, Culture=neutral, PublicKeyToken=5251f5c14327d259" />
      </bulkCopyProviders>

Here is the AppFabricBulkCopyProvider written in C#.
If you are aware of other ways of implementing this please let me know as I am interested.

Enjoy

    public class AppFabricBulkCopyProvider : IBulkCopy
    {
        public int BatchSize { get; set; }
        public DbConnection Connection { get; set; }
        public string DestinationTableName { get; set; }

        public void WriteToServer(System.Data.IDataReader dataReader)
        {
            System.Diagnostics.Trace.WriteLine("Helper.Provider.WriteToServer");

            try
            {
                AppFabricData appfRequest = new AppFabricData();
                appfRequest.strTable = this.DestinationTableName;
                appfRequest.dtsTable = ConvertDataReadertoDataSet(dataReader);

                appfRequest.dtsTable.WriteXml(string.Format(@"C:\angelol\ MONITORING\{0}-{1}.txt", this.DestinationTableName, Guid.NewGuid().ToString()));

            }
            catch (Exception Exc)
            {
                string strError;

                if (Exc.InnerException == null)
                    strError = string.Format("AppFabric Logging ErrorHandling::Exception:{0}", Exc.ToString());
                else
                    strError = string.Format("AppFabric Logging ErrorHandling::Exception:{0}-InnerException{1}", Exc.ToString(), Exc.InnerException.ToString());

                System.Diagnostics.Trace.WriteLine(strError);
            }
        }


        private DataSet ConvertDataReadertoDataSet(IDataReader reader)
        {
               System.Diagnostics.Trace.WriteLine("Helper.Provider.ConvertDataReadertoDataSet");

            DataSet ds = new DataSet();
            DataTable dataTable = new DataTable(this.DestinationTableName);

            DataTable schemaTable = reader.GetSchemaTable();


            DataRow row;
            string columnName;
            DataColumn column;

            int count = schemaTable.Rows.Count;

            for (int i = 0; i < count; i++)
            {
                row = schemaTable.Rows[i];
                columnName = (string)row["ColumnName"];
                column = new DataColumn(columnName, (Type)row["DataType"]);
                dataTable.Columns.Add(column);
            }
            ds.Tables.Add(dataTable);

            object[] values = new object[count];

            try
            {
                dataTable.BeginLoadData();
                while (reader.Read())
                {
                    reader.GetValues(values);
                    dataTable.LoadDataRow(values, true);
                }

            }
            finally
            {
                dataTable.EndLoadData();
                reader.Close();
            }

            //return dataTable;
            return ds;
        }
    }