Monday, July 5, 2010

Behind The .locals init Flag

As we all probably know, the C# language specification demands that every declared local variable will be assigned-to before its usage.
Even so, whoever used ILDASM to peek into the compiler's generated IL code, must have noticed that right after the deceleration for a method, there's some peculiar line that starts with .local init. Actually something like that:

.method private hidebysig static void  Main(string[] args) cil managed
{
  .entrypoint
  // Code size       10 (0xa)
  .maxstack  1
  .locals init ([0] int32 x) <--- localsinit flag 
  IL_0000:  ldc.i4.4
  IL_0001:  stloc.0
  IL_0002:  ldloc.0
  IL_0003:  call       void [mscorlib]System.Console::WriteLine(int32) 
}

This line represents the existence of the CorILMethod_InitLocals flag in the current method's header. This flag effectively guarantees that the CLR will initialize all of the local variables declared in the method's body to they're default values. Meaning, regardless to which default value you have chosen to set your local variable to (in our case, the variable x gets the value of 4), the platform will always make sure that before our code will execute, the local variable x will necesserily be initialized to its legal, default value (in this case, 0).

In Microsoft's implementation of the CLI, this flag always exists in the method's header (assuming there are local variables declared in its body). This could make us wonder why would the compiler insist on reporting an error every time a programmer forgets the initialize its local variables before using them. This constraint held by the compiler may seem redundant, but in fact, there's several reason to why it is quite necessary.

Before diving into the meaning of the .locals init flag, let's review again the issue of the so called "duplicate assignment" performed by the compiler and the CLR.
Looking at the IL code from before, one may think that every time we declare a local variable we have to pay some "initialization overhead" since both the compiler and the CLR insist on initializing it to some "default value". Even though this overhead is quite minimal, it still gives us some "bad vibe" since it just seems redundent.
But in fact, this duplicate assignment never really occurs. The reason for that lays inside the way that the .locals init flag guarantees the default values assignment. All it does, is to make sure the JIT compiler will generate code that will initialize the variable before its usage. In our case, the JIT will have to generate a mov instruction that will set our x variable to 0.
And so, the assembler code we get during run-time (without using optimizations), confirms it:

Normal JIT generated code
ConsoleApplication4.Program.Main(System.String[])
Begin 00e20070, size 30
00E20070 push        ebp
00E20071 mov         ebp,esp
00E20073 sub         esp,8
00E20076 mov         dword ptr [ebp-4],ecx
00E20079 cmp         dword ptr ds:[00942E14h],0
00E20080 je          00E20087
00E20082 call        7A0CA6C1 (JitHelp: CORINFO_HELP_DBG_IS_JUST_MY_CODE)

-------------------- Generated code due to the LocalsInit flag  ----------------
00E20087 xor         edx,edx                 // zero out the EDX register
00E20089 mov         dword ptr [ebp-8],edx // assign the value of EDX to the location of 'X'

--------------------- Our own application's code ---------------------------------
00E2008C mov         dword ptr [ebp-8],4   // assign the value 4 to the location of 'X'

00E20093 mov         ecx,dword ptr [ebp-8]
00E20096 call        79793E74 (System.Console.WriteLine(Int32), mdToken: 060007c3)
00E2009B nop
00E2009C mov         esp,ebp
00E2009E pop         ebp
00E2009F ret

If so, in this code example one can defenitly see the effect of the localsInit flag on the generated assembler code. And also, we can see the existence of the "duplicate-assignment" phenomena as mentioned before.
However, one should remember that this code was generated without any usage of JIT optimizations. Once we will allow these optimizations, we would see that the JIT is able to identify the duplicate assignment, and treats it as dead code since it has no effect on the variable. As a result, the first assignment is completely removed, and only the user's value initialization appears in the generated code:

Normal JIT generated code
ConsoleApplication4.Program.Main(System.String[])
Begin 00c80070, size 19
00c80070 push    ebp
00c80071 mov     ebp,esp
00c80073 call    mscorlib_ni+0x22d2f0 (792ed2f0) (System.Console.get_Out(), mdToken: 06000772)
00c80078 mov     ecx,eax
00c8007a mov   edx,4  // assign 4 to the "virtual representation" of X 
00c8007f mov     eax,dword ptr [ecx]
00c80081 call    dword ptr [eax+0BCh]

The first thing we notice is that we don't have a representation of x in our application's main memory. Instead, there's a CPU register that holds its "virtual" representation. But more importantly, we can now see that there's no remnant of the previous duplicate assignment we've witnessed due to the usage of the localsInit flag.

Now, we can check what in fact is the meaning behind the usage of localsInit and the compilers constraint for initializing local variables before they're usage.
Microsoft's argument regarding using the Definite Assignment mechanism is that most of the times programmers don't to initialize they're local variables are cause by logical bugs, and not by the fact that the programmer relies on its compiler to set the variable to its default value. In one of Eric Lippert's comments on his blog, he says it himself:
"The reason we require definite assignment is because failure to definitely assign a local is probably a bug. We do not want to detect and then silently ignore your bug! We tell you so that you can fix it."
The importance of the localsInit flag, can be summarized in one word: Verification.
Verification is the process in which the CLI makes sure that all of the CIL code that exit in the application is "safe". This includes making sure that all of that methods we use are able the receive the number of parameters we pass them, the parameter's types, that all of the local variables are properly initialized, etc.
In case the CLR detects a piece of code that fails the verification process, a VerficationException will be thrown.
Nontheless, not every IL code needs to be verifiable, as mentioned in Partition III of the standard:
"It is perfectly acceptable to generate correct CIL code that is not verifiable, but which is known to be memory safe by the compiler writer. Thus, correct CIL  might not be verifiable, even though the producing compiler might know that it is memory safe."
Having said that, every time we write some unverifiable code, we have to update the proper permissions using the SecurityPermissionAttribute, and explicitly tell the CLR not the perform verification on our code using the SkipVerfication property (the CLR won't perform definite assignment analysis on our code). One of the most common times in which we would want to this is when we write unsafe code. In such cases, we will have to explicitly mark the required check-box in the project's properties, thus, allowing to compiler to compile our unsafe code blocks, and making it to add the UnverifiableCodeAttribute to the generated assembly, telling the CLR that this module is unverifiable.

The verification process requires that every local variable will be initialized. To be exact, it requires that in case no one requested to skip over the verification process, the localsInit flag must be present in the generated IL. For this reason, looking at the CIL instructions reference, you may encounter remarks such as this:
"Local variables are initialized to 0 before entering the method only if the localsinit on the method is true (see Partition I) ... System.VerificationException is thrown if the the localsinit bit for this method has not been set, and the assembly containing this method has not been granted
System.Security.Permissions.SecurityPermission.SkipVerification (and the CIL does not perform automatic definite-assignment analysis) "
Later on, the document address the overhead issue of performing the definite assignment analysis during runtime:
"Performance measurements on C++ implementations (which do not require definite-assignment analysis) indicate that adding this requirement has almost no impact, even in highly optimized code. Furthermore, customers incorrectly attribute bugs to the compiler when this zeroing is not performed, since such code often fails when small, unrelated changes are made to the program."

2 comments:

  1. It's worth mentioning, though, that verification almost never takes place these days, since all programs running on the local machines are exempted from it.

    I guess it's a good thing, since the compiler still has several bugs which all lead to it producing unverifiable code. This is still true as of VS 2012.

    Verification seems like a thing people talking about .NET like to mention, but a thing that's just not used. At all. Sadly. Who needs type safety.

    ReplyDelete
  2. However, one should remember that this code was generated without any usage of JIT optimizations. Once we will allow these optimizations, we would see that the JIT is able to identify the duplicate assignment, and treats it as dead code since it has no effect on the variable. As a result, the first assignment is completely removed, and only the user's value initialization appears in the generated code:

    I get double assignment:

    push rbx
    push rdi
    sub rsp,38h
    mov dword ptr [rsp+20h],0
    mov dword ptr [rsp+20h],11223344h
    mov edx,ecx
    test rdx,rdx
    jle 0000000000000040
    xor ecx,ecx
    ...

    It's a code compiled in release mode with optimizations turned on, and launched outside of Visual Studio.

    ReplyDelete