Andrew Shapira

At least until fairly recently, and possibly still, a lot of things prevent calls from being inlined. For example, if a called method includes struct arguments, or compiles to greater than 32 bytes of IL code, then calls to the method will not be inlined.

The best explanation for this that I've seen is that the JIT compiler needs to execute quickly, and so cannot usually spend a lot of time doing optimizations. If this means that the JIT compiler can't do much inlining, then the current inlining restrictions make sense if inlining is slow relative to the time that's available for JITting.

Could more time-consuming inlining be done elsewhere, e.g., in the back end of csc (This could be shared across different language compilers if needed.)



Re: Common Language Runtime inlining

TaylorMichaelL

Inlining is trickier than it looks. There are a lot of little issues that you generally don't think about when dealing with inlining. Hence the need for restrictions. The fact that a function has a looping structure is less about the code complexity and more about variable and register management. Switch statements require labels which really complicate things.

I'm not convinced that inlining is as important an optimization these days as it use to be. The reason inlining was invented was to get around the overhead of making and returning from a function call. In the earlier days this was expensive. So, at the cost of more memory, compilers would inline simple functions. On modern processors the cost for a function call isn't that high. Given the layout of IL the need to build a full stack frame is also not necessarily required (or could be optimized to not require it). Some of this started back in the 90's with compilers generating different prologue/epilogue code for certain functions to avoid things like backing up all the registers. Since register allocation and whatnot occurs during compilation from IL to native rather than source code to IL the need for inlining is less critical for the compiler. The JIT can still inline if it wants.

The cost-to-benefits for inlining seems small. If a (full stack frame) can be built in, say, 50 cycles on a modern CPU then the JIT would have to be able to determine in less than 50 cycles (per call) that inlining is possible in order for it to save any time. So, if the function was called twice the JIT would have to be able to decide in less than 100 cycles that inlining was viable otherwise it wasted more cycles then it saved. Modern processors (say Pentium+) have instructions to greatly simplify prologue/epilogue code.

Inlining also starts to lose any benefits when you throw in jumps of any sort (when taking CPU pipelining into account). Since jumps are common in most control flow code they are generally excluded which makes most functions unqualified for inlining anyway. Jumping causes processors to lose their pipeline and, thus, slow down the processing. Some processors do pipeline analysis for jumps but they can't handle more than a couple at a time. Add to that the instrutions themselves and inlining really starts to lose its shine. For example a function that requires 10 instructions is called 20 times in a row. If this function is inlined it would require 200 instructions in the pipeline whereas if it weren't then the instruction would require 10 instructions along with the prologue/epilogue code (if any). The processor can better optimize the pipeline with 10 instructions than with 200. There is no way for the processor to know that the same 10 instructions are being repeated over and over again. IMHO

Michael Taylor - 10/24/07

http://p3net.mvps.org





Re: Common Language Runtime inlining

Andrew Shapira

Michael, you raise some good points. Although, I do not agree with some of the things you say, and there are many points in favor of inlining that you did not mention.

But this is not really what I am asking about. I do not intend this thread to be a referendum on inlining, and I do not want a back and forth discussion about the merits of a particular aspect of inlining. What I want to know is whether, technically, philosophically, or otherwise, it would make sense for compiler developers to consider doing at least some inlining in a different phase of compilation than the JIT, e.g., the back end of csc. I have not seen this question addressed elsewhere. Surely this question arose early in the development of the .NET and Mono C# compilers. I'd be interested in hearing about the discussion of this that occurred at that time.

It is true that if you do believe that inlining is not worthwhile, you would consider the answer to be "No, it is not worthwhile to do inlining somewhere other than the JIT." But clearly, many in the developer community perceive that inlining is worthwhile in at least some cases, and it'd be nice to see my original question answered in the context of the assumption that inlining is worthwhile. That is something that will provide readers of this forum with useful information. A discussion of whether inlining is worthwhile is unlikely to cover new ground, unless the original compiler developers really believe that the current extent of inlining is the best possible tradeoff with current technology. Such a belief seems unlikely to me, but I suppose it's possible.





Re: Common Language Runtime inlining

TaylorMichaelL

Perhaps I didn't clarify my justification in the posting. The point wasn't that inlining serves no purpose but that inlining within the compiler itself is not useful. Looking at it from a compiler/language perspective the only routines that could be inlined are private or internal methods of an assembly. Public/Protected/Protected Internal members can not be inlined without requiring that all languages support inlining, which they don't. Therefore modifying a compiler to support a small subset of methods is not that useful. It is better to allow the JIT to handle inlining when it thinks appropriate. This also allows the JIT to potentially inline all methods rather than private/internal. Of course the inline calculation must be done during JITting each time since this information can't be cached anywhere.

Therefore, in answer to your initial poll. I don't see any benefit in implementing inlining in a compiler (unless the compiler already supports inlining directly). It is better to allow the JIT to use its knowledge of the system and any optimizing compiler for the target machine.

Michael Taylor - 10/24/07

http://p3net.mvps.org





Re: Common Language Runtime inlining

Andrew Shapira

What prevents inlining a given call to a public method




Re: Common Language Runtime inlining

Feng Chen - MSFT

Hi Andrew,

I think what you want is preventing the JIT from inlining your class¡¯s methods, please check out MarshalByRefObject Class of which primary function is to suppress inlining by the JIT compiler, thereby allowing the transparent proxy to do its magic. When the JIT compiler tries to inline a method, it first checks to see whether the method¡¯s class derives from System.MarshalByRefObject. If it does, then no inlining will take place.

Hope this helps!

Thanks!






Re: Common Language Runtime inlining

Andrew Shapira

I (like many others) want more inlining, not less.





Re: Common Language Runtime inlining

Feng Chen - MSFT

Hi Andrew,

So far as I know, CLR does not expose any functionality like you said. This will controlled by the JIT, while you can do nothing to control this procedure.

Thanks!






Re: Common Language Runtime inlining

Andrew Shapira

If you read my post, you may see that I was asking about possible changes to the compilation system, and about historical information about design decisions about where to inline and not inline. I don't know why this is marked as an answered question in the forums. I still do not have anything close to the information I was looking for.





Re: Common Language Runtime inlining

Feng Chen - MSFT

Hi Andrew,

Sorry for the misunderstanding, I can follow you now. Recommended reading the following references about this issue:

¡¤ CLR Design Choices

¡¤ JIT Optimizations, Inlining, and Interface Method Dispatching

¡¤ Shared Source CLI Essentials

Hope this helps!

Thanks!






Re: Common Language Runtime inlining

Muhammad Adel

One of the problems with inlining is choosing what to inline.

For example, if there exists a function A that frequently calls function B, and function B is relatively small, then the compiler should inline function B.

But if A calls B based on a user input, the compiler cannot analyze the cost vs benifits of inlining because it depends on how many times A calls B.

Also if Function A calls either function B or C based on user input, the compiler has to choose one of them to inline (the one which is called more frequently), which is something that cannot be decided at runtime.

The problem of needing runtime data to specify optimization technique used can be applied to other optimization points other than function inlining.

This problem is solved partially with the Profile Guided Optimization (PGO) in Visual C++ 2005. You build a special version of the application that contains probes. These probes are generated by the compiler and they record code paths and the number of times these paths are taken. You give this version to your customer and ask him to use it in the same way with the same data he is going to use with the final version. After that you return these extracted data to the compiler which uses them to build an optimized version for this client specifically. The inlining strategy is one of the things that the compiler decides based on these data. Sql server 2005 was developed using this technique.

The problem here is that you have to build a specific version for every user depending on the his style of use and data he enters to the application. Also the user may change the data ranges or style of use for the application from time to time.

Here comes the idea of on the fly PGO. Theoritically the idea is old ( IIRC, I have read about it in books written in the eighties discussing JIT compilation ). The idea is to use the fact that the application is compiled at run time to make profiling at runtime and recompile the application- or some methods- based on these profiling data. In Microsoft's Middle east developers conference that was held in Egypt 2 years ago, it was mentioned that Microsoft intendes to implement this idea in .NET framework 3.0. I don't know if this intention is still in action.

So I think that if version 3 of the framework already has on the fly PGO, you don't need to force inlining at the csc side anymore.






Re: Common Language Runtime inlining

Andrew Shapira

The most interesting statement so far is Michael's, which I asked about earlier:

Public/Protected/Protected Internal members can not be inlined without requiring that all languages support inlining, which they don't.

As I mentioned earlier in this thread, I'd like to better understand the above statement - specifically, what prevents public members from being inlined - and why is that language-dependent (Incidentally, it could be helpful to have csc inline private and internal calls in a less restrictive way than the JIT does now, e.g., have csc inline some calls to methods with struct arguments. I agree though that it would be better to handle inlining in a unified way, and it does seem a bit ugly to have csc inline calls to private and internal methods but not calls to public methods.)

Muhammad - unless PGO inlining also involves changing the existing restrictions on what can be inlined, PGO inlining is a side issue here. What I want to understand is whether the framework developers (e.g., Microsoft and Mono) will ever ease the existing inlining restrictions, or whether those restrictions are likely to remain in place more or less permanently. And for whichever case is true, I'd like to understand why.

Feng - the CLR Design Choices reference does not seem to help with this question. The second reference states in the very first paragraph that the shared source implementation of the JIT is a naive fast JIT, so the Shared Source CLI Essentials book seems not worthwhile to obtain for the purpose of answering my questions about inlining. The second reference appears to contain a lot of useful information, and I plan to eventually read it in detail. Skimming through it suggests that it doesn't address what I was asking about, though.

To reiterate: I'd like to know whether it might make sense sense to change csc to do less restrictive inlining than the JIT, and why or why not.

By the way - why do the same responses in this thread keep getting marked as Answers I keep having to go unmark them. (Also I do not like the choice between "unhelpful" and "answer" - some responses may be helpful but not answer the question.)