GET Compiling and decompiling try-with-resources / Sudo Null IT News FREE

Compiling and decompiling try-with-resources, Beaver State a tarradiddle about how I firm a bug and what came of it.

Introduction

Some time ago, the backlog of a working draft copy was almost empty, and different research tasks surfaced. One of them sounded very intriguing: to fasten modification testing to the fancy victimisation PITest . Connected Habré there is already a rattling detailed review of this depository library (with examples and pictures). I will not recite this article in my own words, but still I urge that you familiarize yourself with information technology front.

I hold that the idea of ​​mutational testing, I fired up. With almost no extra drive, acquiring a puppet for determination potentially dangerous places in your computer code is worth it! I at once got low to byplay. At that time, the library was relatively young, and as a result, very crude: here you need to play a little with the maven configuration, and there - patch the hack-in for Sonar. Nevertheless, after a while I was still healthy to verify the entire project. Result: hundreds of surviving mutations! Scale evolution on our form server.

Rolling rising my sleeves, I plunged into work. In some tests, there is non enough verification of stubs, in others, instead of logic, it is generally not clear what is being tested. Edit out, improve, rewrite. In general, the march began, but the number of surviving mutations did not decrease as rapidly as we craved. The reason was simple: PIT gave a Brobdingnagian amount of false positives on the essay-with-resources hinder . A short search showed that the bug is known , but ease not determinate. Well, the library code is open. Why not persuade him and see what is the matter?

We infer the reasons

I threw a swordlike instance , a building block try out for it and ran PITest. The result is before you: instead of matchless, there are 11 mutations that survived, ten of which argue a line with the "}" symbol. Calls to the tightlipped and addSupressed methods suggest that this line includes the code generated for the try out-with-resources block. To confirm this conjecture, I decided to decompile the class single file. To DO this, I used the JD-Graphical user interface , although I would now commend the built-in IntelliJ Estimation 14 decompiler .

          unexclusive static void primary(String[] args) throws IOException {     ByteArrayOutputStream baos = new ByteArrayOutputStream();     Throwable var2 = null;     try {         baos.flush();     } catch (Throwable var11) {         var2 = var11;         throw var11;     } finally {         if (baos != null) {             if (var2 != null) {                 try {                     baos.close();                 } catch (Throwable var10) {                     var2.addSuppressed(var10);                 }             } else {                 baos.conclusion();             }         }     } }        

The guess was official, merely the interrogative sentence remained: how did two try-with-resources lines turn into a cardinal try-catch-finally lines? gvsmirnov bequeathed to us in any incomprehensible state of affairs to download OpenJDK sources . This is what I did.

All encode related to the try-with-resources compilation tax is located between lines 1428 and 1580 of the Lower form . Javadoc tells us that this class is designed to translate syntactic sugar: no magic, only the simplest modifications to the syntax tree. Everything is in accordance with JLS 14.20.3 .

We figured come out of the closet the compiler behavior. It corpse to understand wherefore the library is trying to mutate the inscribe generated past the compiling program and how it full treatmen. Rummaging done the source, I found KO'd the pursuing. PITest only manipulates bytecode loaded into RAM. It replaces the instructions according to certain rules, and then runs unit tests. For work with bytecode ASM is used .

The number 1 idea was to intercept the line act of the class method visitGeneratedTryCatchBlock MethodVisitor , and then fair tell the library what line should constitute ignored. Similar functionality has already been enforced.for finally block. However, I was surprised to learn that the visitGeneratedTryCatchBlock method does not exist. ASM does not distinguish between the code generated by the encyclopedist and the encode generated by the programmer. Ambush. I had to look into the bytecode, the output and formatting of which was kindly provided by Textifier .

The bytecode of the main method of the TryExample separate

              // access flags 0x9 public static main([Ljava/lang/String;)V throws java/io/IOException    TRYCATCHBLOCK L0 L1 L2 java/lang/Throwable   TRYCATCHBLOCK L3 L4 L5 java/lang/Throwable   TRYCATCHBLOCK L3 L4 L6 null   TRYCATCHBLOCK L7 L8 L9 java/lang/Throwable   TRYCATCHBLOCK L5 L10 L6 nix  L11   LINENUMBER 12 L11   NEW java/Io/ByteArrayOutputStream   DUP   INVOKESPECIAL coffee/io/ByteArrayOutputStream.                  ()V   ASTORE 1  L12   ACONST_NULL   ASTORE 2  L3   LINENUMBER 13 L3   ALOAD 1   INVOKEVIRTUAL java/io/ByteArrayOutputStream.flush ()V  L4   LINENUMBER 14 L4   ALOAD 1   IFNULL L13   ALOAD 2   IFNULL L14  L0   ALOAD 1   INVOKEVIRTUAL Java/io/ByteArrayOutputStream.boon ()V  L1   GOTO L13  L2  FRAME FULL [[Ljava/lang/String; java/io/ByteArrayOutputStream java/lang/Throwable] [Java/lang/Throwable]   ASTORE 3  L15   ALOAD 2   ALOAD 3   INVOKEVIRTUAL java/lang/Throwable.addSuppressed (Ljava/lang/Throwable;)V  L16   GOTO L13  L14  FRAME SAME   ALOAD 1   INVOKEVIRTUAL java/Io/ByteArrayOutputStream.close ()V   GOTO L13  L5   LINENUMBER 12 L5  FRAME SAME1 java/lang/Throwable   ASTORE 3   ALOAD 3   ASTORE 2   ALOAD 3   ATHROW  L6   LINENUMBER 14 L6  FRAME SAME1 java/lang/Throwable   ASTORE 4  L10   ALOAD 1   IFNULL L17   ALOAD 2   IFNULL L18  L7   ALOAD 1   INVOKEVIRTUAL coffee/io/ByteArrayOutputStream.close ()V  L8   GOTO L17  L9  FRAME FULL [[Ljava/lang/String; coffee/io/ByteArrayOutputStream java/lang/Throwable T java/lang/Throwable] [Java/lang/Throwable]   ASTORE 5  L19   ALOAD 2   ALOAD 5   INVOKEVIRTUAL java/lang/Throwable.addSuppressed (Ljava/lang/Throwable;)V  L20   GOTO L17  L18  FRAME One   ALOAD 1   INVOKEVIRTUAL java/Io/ByteArrayOutputStream.close ()V  L17  FRAME Unvaried   ALOAD 4   ATHROW  L13   LINENUMBER 15 L13  FRAME FULL [[Ljava/lang/Chain;] []   Generate  L21   LOCALVARIABLE x2 Ljava/lang/Throwable; L15 L16 3   LOCALVARIABLE x2 Ljava/lang/Throwable; L19 L20 5   LOCALVARIABLE baos Ljava/io/ByteArrayOutputStream; L12 L13 1   LOCALVARIABLE args [Ljava/lang/String; L11 L21 0   MAXSTACK = 2   MAXLOCALS = 6                          

The naive supposal that the try-bewitch-finally draw a blank is enforced at the JVM level has not been confirmed . There is nobelium specialized instruction for it, only an exception postpone and goto between labels. It turns out that past modular means information technology will not sour to recognize the generated obturate. Need to search another solution.

What if…

Before starting to guess on coffee grounds, I decided to put bytecode marks on the decompiled class. That's what came out of it.

          national static nullity independent(String[] args) throws IOException {     ByteArrayOutputStream baos = new ByteArrayOutputStream(); // L11     Throwable primaryExc = null; // L12     prove {         baos.purge(); // L3     } watch (Throwable t) { // L5         primaryExc = t;         throw t;     } finally { // L6         if (baos != null) { // L4 L10             if (primaryExc != void) {                 adjudicate {                     baos.close(); // L0 L7                 } catch (Throwable suppressedExc) { // L2 L9                     primaryExc.addSuppressed(suppressedExc); // L15 L19                 } // L1 L16 L8 L20             } else {                 baos.finale(); // L14 L18             }         } // L17     } // L13 }        

Two main ways of platform execution are clearly nascent:

L11 L12 L3 {L4 [L0 (L2 L15 L16) L1] L14} L13 L11 L12 L3 [L5 {L6] L10 [L7 (L9 L19 L20) L8] L18 L17}        

Underneath from each one other are labels whose code blocks match or about match. In parentheses is the code that volition live executed when the closemouthed method acting throws an exception. Likewise in guileless - when the even out method acting . Two shipway upset out because the last block was substituted by the compiler twice. Well, now, to completely break your visual parser: the labels in frizzly brackets refer to line 11. The false line PITest refers to the unvarying line.

Here is the root! A minimally repetition prepare of instructions should be highlighted. If much a jell is found in the bytecode being proved, and even on cardinal production line, the generated code for the try on-with-resources closure is available. It doesn't sound very hard, but I decided to give it a try. To a lower place is a list of operating instructions that I complete in the lead with.

          private static final Heel              JAVAC_CLASS_INS_SEQUENCE = Arrays.asList(   ASTORE, // store throwable   ALOAD, IFNULL, // closeable != zipp   ALOAD, IFNULL, // localThrowable2 != null   ALOAD, INVOKEVIRTUAL, GOTO, // closeable.fine()   ASTORE, // Throwable x2   ALOAD, ALOAD, INVOKEVIRTUAL, GOTO, // localThrowable2.addSuppressed(x2)   ALOAD, INVOKEVIRTUAL, // closeable.warm()   ALOAD, ATHROW); // have throwable                  

Something like this can be mapped to code in a finally block.

          } at length {     if (closeable != null) { // IFNULL         if (localThrowable2 != null) { // IFNULL             try {                 closeable.close(); // INVOKEVIRTUAL or INVOKEINTERFACE             } snap (Throwable x2) {                 localThrowable2.addSuppressed(x2); // INVOKEVIRTUAL             }         } else {             closeable.skinny(); // INVOKEVIRTUAL or INVOKEINTERFACE         }     } } // ATHROW        

"Non so difficult," I thought after several days of hard work. Threw few more examples ; wrote tests that use them. Everything is fine, everything works. I dependable to build PITest to run it connected animate encipher: the tests cruel. Not the ones I wrote; others.

Compilers are different

Indeed, the code has moved from the "does not compile" stage to the "does not work" represent. One of the tests that existed before this roughshod. Rolled back - it works. Inside the test, the Java7TryWithResources.class.binful single file that was already in the project is checked . Having printed the bytecode, I could not believe my eyes: a completely different order of instructions was used to compile effort-with-resources!

Difficult non to panic, I began to check completely the compilers at hand. I worked with javac from Oracle JDK, javac from OpenJDK expectedly gave a look-alike leave. I tried different versions: to nobelium service. Information technology was the turn of compilers that were not at hand. Eclipse Compiler for Coffee, ECJ. Compiled, written bytecode - at first glance it looks like-minded the unmatched I'm looking.

The bytecode of the main method of the TryExample by ECJ grade

                              // access flags 0x9  public static main([Ljava/lang/String;)V throws Java/io/IOException     TRYCATCHBLOCK L0 L1 L2 null    TRYCATCHBLOCK L3 L4 L4 null   L5    LINENUMBER 12 L5    ACONST_NULL    ASTORE 1    ACONST_NULL    ASTORE 2   L3    NEW coffee/io/ByteArrayOutputStream    DUP    INVOKESPECIAL java/io/ByteArrayOutputStream.                  ()V    ASTORE 3   L0    LINENUMBER 13 L0    ALOAD 3    INVOKEVIRTUAL java/Io/ByteArrayOutputStream.heyday ()V   L1    LINENUMBER 14 L1    ALOAD 3    IFNULL L6    ALOAD 3    INVOKEVIRTUAL Java/io/ByteArrayOutputStream.cozy ()V    GOTO L6   L2   Skeletal system FULL [[Ljava/lang/String; java/lang/Throwable java/lang/Throwable coffee/Io/ByteArrayOutputStream] [java/lang/Throwable]    ASTORE 1    ALOAD 3    IFNULL L7    ALOAD 3    INVOKEVIRTUAL java/io/ByteArrayOutputStream.end ()V   L7   FRAME CHOP 1    ALOAD 1    ATHROW   L4   FRAME SAME1 java/lang/Throwable    ASTORE 2    ALOAD 1    IFNONNULL L8    ALOAD 2    ASTORE 1    GOTO L9   L8   Build Indistinguishable    ALOAD 1    ALOAD 2    IF_ACMPEQ L9    ALOAD 1    ALOAD 2    INVOKEVIRTUAL java/lang/Throwable.addSuppressed (Ljava/lang/Throwable;)V   L9   FRAME SAME    ALOAD 1    ATHROW   L6    LINENUMBER 15 L6   FRAME Hack 2    RETURN    MAXSTACK = 2    MAXLOCALS = 4                          

After that, I distinct to decompile the resulting classify file. The solvent of the decompiler refused to compile hindmost. Recovered, nothing, you tail end already work with this. Hands bringing the program code in accordance of rights with the bytecode, I got the shadowing.

          unexclusive static void main(String[] paramArrayOfString) throws Throwable {     Throwable primaryExceptionVariable = null; // L5     Throwable caughtThrowableVariable = null;     try {         ByteArrayOutputStream baos = new ByteArrayOutputStream(); // L3         try on {             baos.flush(); // L0         } watch (Throwable t) {             primaryExceptionVariable = t; // L2             throw primaryExceptionVariable; // L7         } finally {             if (baos != null) { // L1                 baos.close();             }         }     } catch (Throwable t) {         caughtThrowableVariable = t; // L4         if (primaryExceptionVariable == null) {             primaryExceptionVariable = caughtThrowableVariable;         } other if (primaryExceptionVariable != caughtThrowableVariable) { // L8             primaryExceptionVariable.addSuppressed(caughtThrowableVariable);         }         throw primaryExceptionVariable; // L9     } // L6 }        

ECJ takes a completely different approach to compiling try-with-resources. Labels are observably small, blocks of code are observably larger. Instead of a bloated table, exceptions are simply thrown to the next layer. In the more complicated examples, you dismiss notice that it turns out a sort of nesting doll.

What is under the hood? I again went to download the informant, this time ECJ . Compilation of the try assertion is obscure in the TryStatement file . This time zero trees, only opcodes, only hardcore. The bytecode responsible for try-with-resources is generated between lines 500 and 604. From the history of commits, it is clearly seen that the body of the try block was simply framed by a range of mountains of calls to produce and close resources.

Because if there is no substitution of the finally embarras, then there is no duplication of encrypt. However, due to nesting, the same actions are perennial for assorted exceptions. I took advantage of this. The set of instructions for ECJ is as follows.

          private atmospherics final List              ECJ_INS_SEQUENCE = Arrays.asList(   ASTORE, // shop throwable2   ALOAD, IFNONNULL, // if (throwable1 == null)   ALOAD, ASTORE, GOTO, // throwable1 = throwable2;   ALOAD, ALOAD, IF_ACMPEQ, // if (throwable1 != throwable2) {   ALOAD, ALOAD, INVOKEVIRTUAL, // throwable1.addSuppressed(throwable2)   ALOAD, ATHROW); // throw away throwable1                  

And then the same java code looks like.

          if (throwable1 == null) { // IFNONNULL     throwable1 = throwable2; } else {     if (throwable1 != throwable2) { // IF_ACMPEQ         throwable1.addSuppressed(throwable2); // INVOKEVIRTUAL     } } // ATHROW        

What about the rest of the compilers? IT turned out that AspectJ generates almost the same bytecode equally ECJ. For him, there was no need to make up a separate sequence. I could not download the encyclopaedist from IBM (and I didn't actually want to). Other compilers were ignored due to contemptible preponderance.

results

An attentive reader has already noticed that the set of instructions for javac does non take into account one refinement. To call the methods of the class and the interface, different instructions are actually used: INVOKEVIRTUAL and INVOKEINTERFACE, respectively. The implementation described above takes into account single the first case and does not allow the second. Well, zero, it's not hard to fix.

So what is the result?

First, the main result of the work was a patch that fixes the tease mentioned at the outset of the clause. Almost completely of the code fits in 1 class (not counting the tests), which currently looks like this: TryWithResourcesMethodVisitor . I urge everyone to criticize and propose their unsurpassed options for solving this problem.

Secondly, I found out what are the ways to hoard a stress-with-resources block. As a result, I figured out what try-catch up-finally looks equal at the bytecode level. Good, a spin-off was the translation of an article that I mentioned above in the textbook.

Thirdly, I wrote this article where I told you everything. Perhaps now one of you will be able to increase the fundamental coefficient of squandering victimization the nonheritable knowledge.

And where is the use and morality, you ask? I leave their hunt to the reader. I only note that I enjoyed while piece of writing this article. Hope you got it from reading. See you soon!

PS Equally a bonus, I suggest look early proposals for the implementation of sample-with-resources from Joshua Ernest Bloch.

Stumbled along original ARM block (try-with-resources) proposals, if anyone's curious. V1: https://t.co/Qngv2STN1W , V2: https://t.co/YiR1RvyZWg

- Joshua Bloch (@joshbloch) June 13, 2015

It looks funny.

          {     net LocalVariableDeclaration ;     mathematician #suppressSecondaryException = false;     try Block arrest (final Throwable #t) {         #suppressSecondaryException = rightful;         cam stroke #t;     } finally {         if (#suppressSecondaryException)             try { localVar.close(); } catch(Exception #ignore) { }         else             localVar.come together();     } }        

DOWNLOAD HERE

GET Compiling and decompiling try-with-resources / Sudo Null IT News FREE

Posted by: rollinsnowlielinuld81.blogspot.com

Share this post