c.oloss.us

In an earlier post, I described a using soap services without a web server. The soap services were hosted by a windows service. One thing I forgot to mention is debugging.

Services are difficult to debug, because it's not so easy to break into the OnStart or OnStop methods. To make my debugging easier, I've implemented a class that simplifies the task:

class DebugServiceBase : ServiceBase
{
    ManualResetEvent evt = new ManualResetEvent( false );

    public void DebugService( )
    {
        Console.TreatControlCAsInput = false;
        System.Console.CancelKeyPress += new ConsoleCancelEventHandler(
           OnCancel );

        Console.WriteLine( "Press CTRL-C to stop." );

        OnStart( null );

        Stream stdIn = Console.OpenStandardInput( );
        StartRead( stdIn );

        evt.WaitOne( );

        OnStop( );
    }

    void StartRead( Stream stream )
    {
        byte[] buffer = new byte[1];
        stream.BeginRead( buffer, 0, 1, new AsyncCallback( OnRead ),
            stream );
    }

    void OnRead( IAsyncResult asyncResult )
    {
        Stream stream = (Stream) asyncResult.AsyncState;

        stream.EndRead( asyncResult );

        if ( !evt.WaitOne( 0, true ) )
            StartRead( stream );
    }

    void OnCancel( object sender, ConsoleCancelEventArgs e )
    {
        evt.Set( );

        e.Cancel = true;
    }
}

All of my services are descended from this class, instead of just ServiceBase. In my Main method, I've got a bit of code that tests the command line for '-start'. Depending on whether this flag is set, Main either does the normal ServiceBase.Run thing, or just executes new ServiceXService( ).DebugService( );.

When DebugService is called, it sets up a handler for ctrl-C, calls the service's OnStart method, and starts a delegate-based loop that swallows keystrokes. When ctrl-C comes through, it stops waiting for keys, and calls OnStop.

On the bonjour-dev mailing list, it recently came up that resolving .local addresses on windows can be pretty slow, because they are first handed to the Tcpip namespace resolver before bonjour's.

On my system, it looks like simply reordering the namespace providers (using NSPTool) doesn't do it. If the 'Tcpip' provider is installed and enabled, I get times of several seconds to resolve a .local name, and when it is not, resolving them is pretty quick.

One way to speed it up would be to disable the Tcpip provider, but that's not really viable, as causes cause lots of problems resolving non-.local names ;)

A better (and more complicated) solution may be to load the bonjour service resolver (mdnsNSP.dll) directly, and just ask it first. Here's some code that does just that. It's a little dirty, and doesn't do much error checking, but it's a start.

GUID mdnsNspGuid = { 0xb600e6e9, 0x553b, 0x4a19, { 0x86, 0x96, 0x33, 0x5e, 0x5c, 0x89, 0x61, 0x53 } };

HINSTANCE hmod = LoadLibrary( L"c:\program files\bonjour\mdnsNSP.dll" );
if ( hmod )
{
    LPNSPSTARTUP startup_proc = (LPNSPSTARTUP) GetProcAddress( hmod, "NSPStartup" );
    LPNSPCLEANUP cleanup_proc = (LPNSPCLEANUP) GetProcAddress( hmod, "NSPCleanup" );
    if ( startup_proc && cleanup_proc )
    {
        NSP_ROUTINE nsp;
        int res = startup_proc( &mdnsNspGuid, &nsp );
        // check res == 0

        WSAQUERYSET query;
        memset( &query, 0, sizeof( query ) );

        // machine name does not require the trailing dot,
        // but it doesn't hurt either
        query.lpszServiceInstanceName = L"machine.local";
        query.dwNameSpace = NS_DNS;

        HANDLE queryHandle;

        // note both the guid and the serviceclassinfo are ignored
        res = nsp.NSPLookupServiceBegin( &mdnsNspGuid, &query, 
            0, LUP_RETURN_ADDR | LUP_RETURN_BLOB, &queryHandle );
        // check res == 0

        // note that mdnsNSP doesn't actually set resultSize, so we'll pick an
        // arbitrary size, probably larger than necessary
        DWORD resultSize = 1500;
        WSAQUERYSET * result = (WSAQUERYSET *) malloc( resultSize );
        memset( result, 0, resultSize );

        res = nsp.NSPLookupServiceNext( queryHandle, 0, &resultSize, result );
        // check res == 0...if it's not, and GetLastError == WSAEFAULT, the buffer's not big 
        // enough, but there's no telling how bit it needs to be.  If GetLastError returns
        // WSASERVICE_NOT_FOUND, then the resolver could have found it but did not (i.e., it's 
        // a .local address, but could not be found).  If GetLastError returns 
        // ERROR_INVALID_HANDLE, then it's not a name the resolver could have found (i.e.,
        // not a .local address)

        if ( !res && result->dwNumberOfCsAddrs >= 0 )
        {
            SOCKADDR_IN * addr = (SOCKADDR_IN *) result->lpcsaBuffer[0].RemoteAddr.lpSockaddr;

            // do something with addr
        }
        else
        {
            // resolve normally
        }

        free( result );

        nsp.NSPLookupServiceEnd( queryHandle );

        cleanup_proc( &mdnsNspGuid );
    }

    FreeLibrary( hmod );
}

Update: Marc Krochmal made a better suggestion on the list...just call DNSServiceQueryRecord instead.

I've set up two new websites, Antarctica Ice and slidetype.com. I plan to add additional pages and content to Antarctica Ice, because I think it's funny. Slidetype.com, on the other hand, is for work.

The idea behind slidetype.com is a good one. It is intended to be a nice little community for users of our software. It would be a place where users of the software could share slide types (the settings used to tell the system how to process different kinds of slides). Unfortunately, my company wants control over it. Therefore, I won't do anything with it on my own, and it's likely to never be developed beyond the current single page.

At the very least, it was nice to use iWeb. It was really just a matter of dragging over the images and entering some text.

After further examination, I think that doing SOAP with the HttpServerChannel and its .NET remoting friends will turn out poorly. Given the difficulty getting it to work right, getting the solution to scale to a point where it is usable for complicated services isn't worth the trouble, and if security and authentication are necessary, it just won't be possible.

This brings me to my primary complaint: SOAP is too complicated. With all the WS-* stuff, it has grown kind of ugly. The .NET implementation is strongly typed, and fairly early bound. It's starting to look a lot like COM, and WSDL is its IDL.

That doesn't change the fact that I need to implement SOAP-based services...

I did some more searching, and I found Web Services Extensions. Microsoft has embraced the complexity of SOAP, and created a relatively simple API around it. Using WSE is easy, but it comes with a price. Because of the complexity of the the standards it implements (WS-*), using WSE to implement a service pretty much guarantees that your service will be compatible only with other WSE clients/servers, or someday, with similarly complex Java clients/servers. Even old .NET soap clients have trouble communicating with WSE services. Once you go WSE, you're stuck with it.

WSE was designed to use IIS and ASP.NET for hosting services via http, but I'm not interested in hosting services with IIS (I don't even want IIS installed on my machines). WSE makes it easy to host services via the "soap.tcp" protocol, but that doesn't sound terribly appealing either. With the HTTP.SYS (under Windows XP SP2, or Windows Server 2003), and a little code, it's possible to self-serve with http, with relatively little difficulty.

Aaron Skonnard wrote an article for MSDN magazine that includes some code for a custom HTTP.SYS-based transport for WSE 3.0. It's good stuff, but I recommend a few changes.

If the http method for the request is GET, the HttpSysAdapter class returns the WSDL for the service, and if that fails, it returns a blob of html detailing the error. It is helpful to set the content type on that response to text/xml before writing to it (do that just after the test for whether the method is GET)...that way, a client can make sense of it.

Also, HttpSysAdapter.BeginReceive uses a nested class (HttpSysAsyncResult) that steals a threadpool thread and calls HttpListener.GetContext. This causes problems when the app is shutting down, since the threadpool thread isn't necessarily marked as a background thread, and shutting down in the middle of GetContext seems to make the whole thing a little angry. It's better to eliminate HttpSysAsyncResult, and have HttpSysAdapter.BeginReceive just call HttpListener.BeginGetContext. It saves a threadpool thread, and makes shutdown easier.

Putting these two changes together, remove HttpSysAsyncResult, and replace HttpSysAdapter.BeginReceive and HttpSysAdapter.EndReceive with these:

/// <summary>
/// This function is used to begin listening via HTTP.SYS.
/// </summary>
/// <param name="callback">Method to call when something is received.</param>
public IAsyncResult BeginReceive( AsyncCallback callback )
{
    this.listening = true;

    return listener.BeginGetContext( callback, this );
}

/// <summary>
/// When the callback fed to <see cref="BeginReceive"/> is called, call <see cref="EndReceive"/>
/// to finish processing the received data and retrieve a <see cref="SoapEnvelope"/> if one came in.
/// </summary>
/// <returns>The received <see cref="SoapEnvelope"/>, or <see langref="null"/> if one did not 
/// come in.</returns>
public SoapEnvelope EndReceive( IAsyncResult result )
{
    HttpSysAdapter adapter = result.AsyncState as HttpSysAdapter;
    if ( adapter == null )
        throw new ArgumentException( "AsyncResult not obtained from HttpSysAdapter.BeginReceive()", "result" );

    // finish retrieving the context...also causes exceptions thrown during async operation
    // to be rethrown here
    HttpListenerContext ctx = adapter.listener.EndGetContext( result );

    // Check the HTTP method used in message, if GET, let's return the WSDL
    if ( ctx.Request.HttpMethod.Equals( "GET" ) )
    {
        ctx.Response.ContentType = "text/xml";

        try
        {
            // See if we can retrieve the receiver for the supplied Url from SoapReceivers
            EndpointReference er = new EndpointReference( ctx.Request.Url );
            if ( !SoapReceivers.Contains( er ) )
                throw new Exception( "Make sure you entered the exact same addressed supplied to SoapReceivers.Add, including the training forward slash '/'" );

            object obj = SoapReceivers.Receiver( er );

            // If the target receiver is in a WsdlEnabledReceiver, call GetDescription and serialize WSDL
            // into the response stream, and we're done
            WsdlEnabledReceiver receiver = obj as WsdlEnabledReceiver;
            if ( receiver != null )
            {
                ServiceDescription desc = receiver.GetDescription( ctx.Request.Url );

                using ( XmlTextWriter tw = new XmlTextWriter( ctx.Response.OutputStream, Encoding.UTF8 ) )
                    desc.ServiceDescriptions[0].Write( tw );
            }
            else
                throw new Exception( "In order to enable retrieving the service description (WSDL) via the browser, " +
                    "you must supply a WsdlEnabledReceiver when calling SoapReceivers.Add" );
        }
        catch ( Exception e )
        {
            // Otherwise, we can't provide the WSDL so return some HTML telling the user what happened
            using ( XmlTextWriter tw = new XmlTextWriter( ctx.Response.OutputStream, Encoding.UTF8 ) )
            {
                tw.WriteStartElement( "html" );
                tw.WriteStartElement( "body" );
                tw.WriteElementString( "h1", "Service description unavailable" );
                tw.WriteElementString( "p", e.Message );
                tw.WriteEndElement( );
                tw.WriteEndElement( );
            }
        }

        // returning a null SoapEnvelope prevents dispatching from continuing
        return null;
    }

    // if it wasn't a GET, we'll assume we need to deserialize the body into a SoapEnvelope
    // using the SoapPlainFormatter (we're not dealing with MTOM here)
    SoapEnvelope env = null;
    ISoapFormatter formatter = new SoapPlainFormatter( );
    try
    {
        env = formatter.Deserialize( ctx.Request.InputStream );
    }
    catch
    {
        // non-soap messages shouldn't crash us - might want to do more error handling here
        // currently, they are simply ignored.
    }

    if ( null != env )
    {
        // Setup WS-Addressing headers
        AddressingHeaders headers = env.Context.Addressing;
            Uri localEndpoint = ctx.Request.Url;
            Uri remoteEndpoint = new Uri( String.Format( "http://{0}", ctx.Request.LocalEndPoint.ToString( ) ) );

        // This does most of the hard work for us
        headers.SetRequestHeaders( new Via( localEndpoint ), new Via( remoteEndpoint ) );

        // save HttpListenerContext in request message - we'll need this later in the
        // HttpSysOutputChannel to transmit the response back down
        env.Context.SessionState.Set<HttpListenerContext>( ctx, "HttpSysAdapter" );
    }

    return env;
}

Of course, now that the calls are no longer in a separate thread, the exceptions thrown by calls to EndReceive are a little different. HttpSysTransport.OnReceiveComplete used to handle an AsynchronousOperationException, but now it needs to be a bit more specific. When the app is being shut down, HttpSysTransport.OnReceiveComplete will be called, and the call to HttpSysAdapter.EndReceive will throw a HttpListenerException, with the ErrorCode set to 995...this is normal, and OnReceiveComplete should just return without handling it. Otherwise, exceptions should be logged, and HttpSysAdpater.BeginReceive should be called again:

try
{
    // retrieve the SoapEnvelope from the HttpSysAdapter
    envelope = adapter.EndReceive( result );

    // start listening again (otherwise, it will stop after one message)
    adapter.BeginReceive( new AsyncCallback( this.OnReceiveComplete ) );
}
catch ( HttpListenerException e )
{
    // this exception is thrown when the thread is ended,
    // with error code 995.  This is a normal part of shutdown,
    // and should be ignored.  Otherwise, the exception should
    // be logged, and we should keep listening.
    if ( e.ErrorCode != 995 )
    {
        EventLog.WriteError( "HttpSysTransport failure: " + e.Message );
        adapter.BeginReceive( new AsyncCallback( this.OnReceiveComplete ) );
    }
    return;
}
catch ( Exception e )
{
    // log any other exceptions
    EventLog.WriteError( "HttpSysTransport failure: " + e.Message );
    adapter.BeginReceive( new AsyncCallback( this.OnReceiveComplete ) );
    return;
}

Hosting web services using HTTP.SYS, WSE, and Aaron Skonnard's code may not be terribly good for cross-platform access, but it's certainly easy, and with these changes, it meets my immediate need.

I have now successfully implemented a soap-based web service without a web server on the .NET 2.0 Framework.

There is plenty of documentation on how to consume web services from .NET, and how to create web services with ASP.NET. There is little documentation of running web services without ASP.NET, and what is available isn't helpful, so I decided to describe my adventures with it...

I initially looked into implementing a service via .NET remoting, using Soap Formatting. This was quite promising, and there was some documentation available for it. I implemented a basic service, as follows:

// this interface is actually defined in a separate assembly,
// intended to be shared between the client and server
public interface IServiceX
{
    int GetNumber( );
}

public class ServiceX : MarshalByRefObject, IServiceX
{
    public int GetNumber( )
    {
        return 291;
    }
}

class ServiceXService : ServiceBase
{
    // create the server channel
    HttpServerChannel serverChannel;

    public ServiceXService( )
    {
        // port 3k
        serverChannel = new HttpServerChannel( "ServiceX Channel", 3000 );
    }

    protected override void OnStart( string[] args )
    {
        // register the server channel
        ChannelServices.RegisterChannel( serverChannel, false );

        // expose an object for remote calls, allowing the .NET framework
        // to manage the object's lifetime (the object will be reused too)
        RemotingConfiguration.RegisterWellKnownServiceType(
            typeof( ServiceX ), "ServiceX", WellKnownObjectMode.Singleton );
    }

    protected override void OnStop( )
    {
        // clean up channel registration
        ChannelServices.UnregisterChannel( serverChannel );
    }
}

Sure enough, this created a web service that used soap over http to access an object of type ServiceX, at the url http://localhost:3000/ServiceX. From .NET, after referencing/importing the assembly that defined IServiceX, a client could access the service as follows:

HttpChannel channel = new HttpChannel( );
ChannelServices.RegisterChannel( channel, false );

string url = "http://localhost:3000/ServiceX";

IServiceX sx = (IServiceX) Activator.GetObject( typeof( IServiceX ), url );
int number = sx.GetNumber( );

Internally, .NET remoting over an HttpChannel is implemented using soap. Unfortunately, it is ugly soap...The namespaces are not pretty, and there is a lot of .NET-specific junk in there, and it is pretty clear that the service is implemented by .NET. The service would not be terribly pleasant to access from a non-.NET language.

The WSDL for the service is automatically available to callers by affixing ?WSDL to the end of the service url. For the above service, at http://localhost:3000/ServiceX, the url would be http://localhost:3000/ServiceX?WSDL (note that the suffix is not case sensitive). I wanted to avoid as many .NET-isms as possible in the SOAP and WSDL (such as the default provided namespaces that include the assembly names, class namespaces, and version numbers). The documentation implies that this can be customized, by decorating the classes involved with attributes.

Figuring out what decorations were necessary was not a painless process. Most of the available documentation for .NET services involves ASP.NET, and as it turned out, none of it applies here. I quickly determined that the attributes used for xml serialization (in the System.Xml.Serialization namespace) and ASP.NET web services (in the System.Web.Services namespace) were completely irrelevant. Only attributes in the System.Runtime.Remoting.Metadata namespace had an effect on the service's generated WSDL.

After an initial try at decorating decorating the interface, I started getting some RemotingExceptions when I ran the client code. The message associated with these exceptions weren't terribly helpful (Invalid SOAPAction specified: ServiceX#GetNumber). It seems that even though the ServiceX class was descended from the IServiceX interface, the attributes on the interface weren't being used during deserialization of the soap message. The problem was easily solved, by decorating the ServiceX class with the same attributes.

This change allowed calls from .NET clients to work, but caused problems when accessing the WSDL. When I tried to retrive the WSDL using the ?WSDL url suffix, it returned error code 500, along with a NullReferenceException. The error was coming from System.Runtime.Remoting.MetadataServices.WsdlGenerator.RealSchemaType.Resolve, a class deep in the bowels of the SdlChannelSink (the class that processes the ?WSDL url, returning the WSDL). Whenever I set an XmlNamespace on the ServiceX class and its interface (using the SoapType attribute), the exception was returned instead of WSDL. Eventually, I determined that this was caused by the WSDL generator failing to properly determine the xml namespace to be used for the ServiceX class, presumably because it was trying (unsucessfully, for some reason) to determine it from the IServiceX interface. The solution was to make ServiceX no longer implement the IServiceX interface explicitly. As the ServiceX class is used in only the service, and the IServiceX interface would be used only by the client, it works out.

I eventually managed to clean up the namespaces and the soap action name by applying the SoapType attribute to the interface, applying the SoapMethod attribute to its members, and not descending from the interface, sort of like the following:

[SoapType(
    XmlElementName = "ServiceX",
    XmlNamespace = "http://example.org/ServiceX" )]
public interface IServiceX
{
    [SoapMethod(
        SoapAction = "ServiceX#GetNumber",
        XmlNamespace = "http://example.org/ServiceX" )]
    int GetNumber( );
}

[SoapType(
    XmlElementName = "ServiceX",
    XmlNamespace = "http://example.org/ServiceX" )]
public class ServiceX : MarshalByRefObject
{
    [SoapMethod(
        SoapAction = "ServiceX#GetNumber",
        XmlNamespace = "http://example.org/ServiceX" )]
    public int GetNumber( )
    {
        return 291;
    }
}

Having the client create a dynamic IServiceX proxy to the service via Activator.CreateInstance works well, but it is likely that I will abandon IServiceX, and use the WSDL.EXE tool to generate a proxy class from the WSDL, and use that instead.

The names of the service and its implementing classes have been changed to protect the innocent.