Ldstr, where are you?

In the meantime I'm working on a Fody add-in, which can validate SQL query if it finds one. Details will be provided soon - now I'd like to describe one issue I had when analyzing MSIL code for it

Internally this add-in scans MSIL instructions in an assembly trying to find ldstr opcode in methods' bodies. There is no magic - I haven't found better way to find strings in the generated code, mostly because strings are not decorated in MSIL in any way. It worked just fine for most of test cases I created. I decided to test it in my side-project against more realistic use cases. The output turned out to be somehow surprising:

Fody: Fody (version 1.29.4.0) Executing
Fody/Stamp:   Starting search for git repository in SolutionDir
Fody/Stamp:   Found git repository in E:\Codenova\Klienci\MikroSystem\vNext\.git\
Fody/QueryValidator:   Found 0 queries to validate.
Fody/QueryValidator:   Trying to get configuration for E:\Codenova\Klienci\MikroSystem\vNext\LicznikNET.vNext.Api\obj\Debug\LicznikNET.vNext.Api.exe
Fody/QueryValidator:   Found configuration file E:\Codenova\Klienci\MikroSystem\vNext\LicznikNET.vNext.Api\app.config
Fody/QueryValidator:   Connection string is Data Source=.\SQLEXPRESS;Initial Catalog=vNext;Persist Security Info=True;User ID=foo;Password=bar;MultipleActiveResultSets=True;

What the heck?! I have almost 100 SQL queries in my code and still it found nothing? Impossibru!

I decided to run dotPeek and do some "visual debugging". Let's say we have following method:

public void Bar()
{
     using (var connection = new SqlConnection())
     {
          connection.Query(@"|> SELECT * FROM dbo.Foo");
     }
}

This works perfectly fine, the generated MSIL is what I expected:

IL_0006: ldloc.0      // connection
IL_0007: ldstr        "|> SELECT * FROM dbo.Foo"
IL_000c: ldnull       
IL_000d: ldnull       
IL_000e: ldc.i4.1     
IL_000f: ldloca.s     V_1
IL_0011: initobj      valuetype [mscorlib]System.Nullable`1<int32>
IL_0017: ldloc.1      // V_1
IL_0018: ldloca.s     V_2
IL_001a: initobj      valuetype [mscorlib]System.Nullable`1<valuetype [System.Data]System.Data.CommandType>
IL_0020: ldloc.2      // V_2
IL_0021: call         class [mscorlib]System.Collections.Generic.IEnumerable`1<object> [Dapper]Dapper.SqlMapper::Query(class [System.Data]System.Data.IDbConnection, string, object, class [System.Data]System.Data.IDbTransaction, bool, valuetype [mscorlib]System.Nullable`1<int32>, valuetype [mscorlib]System.Nullable`1<valuetype [System.Data]System.Data.CommandType>)
IL_0026: pop    

But this doesn't reproduce my problem because in my side-project all my DB calls are asynchronous(using Dapper of course). Fixed example could look like this:

public async Task Foo()
{
     using (var connection = new SqlConnection())
     {
          await connection.QueryAsync(@"|> SELECT * FROM dbo.Foo");
     }
}

Now, when I check MSIL, the root of all evil will be revealed:

IL_0000: ldloca.s     V_0
IL_0002: call         valuetype [mscorlib]System.Runtime.CompilerServices.AsyncTaskMethodBuilder [mscorlib]System.Runtime.CompilerServices.AsyncTaskMethodBuilder::Create()
IL_0007: stfld        valuetype [mscorlib]System.Runtime.CompilerServices.AsyncTaskMethodBuilder QueryValidator.Fody.TestWeb.TestClass/'<Foo>d__0'::'<>t__builder'
IL_000c: ldloca.s     V_0
IL_000e: ldc.i4.m1    
IL_000f: stfld        int32 QueryValidator.Fody.TestWeb.TestClass/'<Foo>d__0'::'<>1__state'
IL_0014: ldloc.0      // V_0
IL_0015: ldfld        valuetype [mscorlib]System.Runtime.CompilerServices.AsyncTaskMethodBuilder QueryValidator.Fody.TestWeb.TestClass/'<Foo>d__0'::'<>t__builder'
IL_001a: stloc.1      // V_1
IL_001b: ldloca.s     V_1
IL_001d: ldloca.s     V_0
IL_001f: call         instance void [mscorlib]System.Runtime.CompilerServices.AsyncTaskMethodBuilder::Start<valuetype QueryValidator.Fody.TestWeb.TestClass/'<Foo>d__0'>(!!0/*valuetype QueryValidator.Fody.TestWeb.TestClass/'<Foo>d__0'*/&)
IL_0024: ldloca.s     V_0
IL_0026: ldflda       valuetype [mscorlib]System.Runtime.CompilerServices.AsyncTaskMethodBuilder QueryValidator.Fody.TestWeb.TestClass/'<Foo>d__0'::'<>t__builder'
IL_002b: call         instance class [mscorlib]System.Threading.Tasks.Task [mscorlib]System.Runtime.CompilerServices.AsyncTaskMethodBuilder::get_Task()
IL_0030: ret  

When implementing my add-in I forgot, that compiler converts all async calls into state machines(or to be more specific - into nested classes implementing IAsyncStateMachine interface). Because I wasn't scanning nested classes(I believe it was a bug anyway), no async query could be found. After loading instructions for nested classes also, I am sure it works as intended:

Fody: Fody (version 1.29.4.0) Executing
Fody/Stamp:   Starting search for git repository in SolutionDir
Fody/Stamp:   Found git repository in E:\Codenova\Klienci\MikroSystem\vNext\.git\
Fody/QueryValidator:   Found 94 queries to validate.
Fody/QueryValidator:   Trying to get configuration for E:\Codenova\Klienci\MikroSystem\vNext\LicznikNET.vNext.Api\obj\Debug\LicznikNET.vNext.Api.exe
Fody/QueryValidator:   Found configuration file E:\Codenova\Klienci\MikroSystem\vNext\LicznikNET.vNext.Api\app.config
Fody/QueryValidator:   Connection string is Data Source=.\SQLEXPRESS;Initial Catalog=vNext;Persist Security Info=True;User ID=foo;Password=bar;MultipleActiveResultSets=True;

Reporting issues in YouTrack automatically

Problem

When working on my side-project for my client, I often require some specific data to address reported issue. It could be a specific request with its body, logged exception with a stack trace or authenticated user's context. I've learned, that expecting someone to paste all necessary data when reporting an error, always fails - remembering each time what you have to include is both tiresome and burdersome. Fortunately there's a possibility to automate things.

Solution

Currently I am using YouTrack so I'll stick to this issue tracker only. YouTrack exposes a REST API, which allows you to easily create, delete and manage reported issues. A reference to this API can be found here. Although it is possible to write your own client, I decided to use YouTrackSharp library - for my expectations it is a reasonable choice.

For now I've been missing following data in most issue reporting cases:

  • lack of full request's body - how to reproduce a problem without knowing what was sent to the server, especially when Jil sometimes goes crazy during deserialization
  • lack of user's context - how to reproduce problem without knowing who exactly performed a request
  • lack of headers sent and all request related data

Because I've already been catching all unhandled exceptions to log them, finding a place to call my "CreateYouTrackIssue" method is a piece of a cake. To be honest, I could take advantage of using EventStore and just raise an event, which will be further proceeded - maybe in the future I recreate it this way.

Whole method could look like this:

private static void CreateYouTrackIssue(Exception ex, Stream body)
{
      using (var reader = new StreamReader(body))
      {
            var conn = new Connection("host", 8080);
            conn.Authenticate("user", "password");

            var json = reader.ReadToEnd();

            dynamic issue = new Issue();
            issue.ProjectShortName = "SOME_NAME";
            issue.Summary = "API error - " + ex.Message;
            issue.Description = string.Format(
                    "An API error occuredd: :\r\nJSON: {0}\r\nStackTrace: {1}",
                    json,
                    ex.StackTrace);
            issue.Assignee = "root";
            issue.Type = "Bug";
            issue.State = "New";

            var im = new IssueManagement(conn);
            im.CreateIssue(issue);

            conn.Logout();
      }
}

This is just a sketch so it doesn't contain all mentioned features but note few things:

  • I'm passing stream to read its content - to get all necessary data passing some request's context would be more practical
  • I'm not handling HTTP connections - YouTrackSharp takes care of it
  • I'm assigning each feature to the root user - you can select whatever user you want

Because Issue() is dynamic, you can pass every custom field you created in YouTrack.

Summary

Above solution has already saved me some time when trying to reproduce reported error. What is more, it makes my client happy - he knows, that each time he sees a red error alert, issue has been created and I have been notified about it.