<html>
<head>
<meta http-equiv="Content-Type" content="text/html; charset=Windows-1252">
<style type="text/css" style="display:none;"> P {margin-top:0;margin-bottom:0;} </style>
</head>
<body dir="ltr">
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="color: rgb(0, 0, 0); font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt;">Firstly, the strings aren't using regex, just a simple search.</span><br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
Here's what I've found:</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
In ReflLogger.constructorMethodInvoke the logic gets the class name using: c.getDeclaringClass().getName()</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
For a lambda this is something like: "<dotted package>.<class>$$Lambda$<count>/<number>"</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
>From my original example, it would be "java.awt.GraphicsEnvironment$$Lambda$1/1480010240"</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
When the class file is written out that final <number> part isn't present, so we end up with just "/java/awt/<span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">GraphicsEnvironment$$Lambda$1.class".
I haven't figured out what happened to <number> along the way, but I do know there are various <span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">dot<->slash transformations which
is how the reflection log ends up with <span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">"</span><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">java.awt.GraphicsEnvironment$$Lambda$1.1480010240"</span></span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><br>
</span></span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">I
made an initial attempt to modify <span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">ReflLogger.constructorMethodInvoke to strip off the "/<number>" from the class name. In my
one sample test this did allow the reflection log's entries to align with the generated .class files and thus soot was able to process the generated reflection log.</span></span></span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><br>
</span></span></span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">Perhaps
with this added information Eric or someone else can give advice on the most robust path forward.</span></span></span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><br>
</span></span></span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">Thanks!</span></span></span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important"><span style="font-family: Calibri, Arial, Helvetica, sans-serif; background-color: rgb(255, 255, 255); display: inline !important">David</span></span></span></span></div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div style="font-family: Calibri, Arial, Helvetica, sans-serif; font-size: 12pt; color: rgb(0, 0, 0);">
<br>
</div>
<div id="appendonsend"></div>
<hr style="display:inline-block;width:98%" tabindex="-1">
<div id="divRplyFwdMsg" dir="ltr"><font face="Calibri, sans-serif" style="font-size:11pt" color="#000000"><b>From:</b> Eric Bodden <eric.bodden@uni-paderborn.de><br>
<b>Sent:</b> Tuesday, April 30, 2019 2:02 AM<br>
<b>To:</b> David Diepenbrock<br>
<b>Cc:</b> soot-list@cs.mcgill.ca<br>
<b>Subject:</b> Re: [Soot-list] Tamiflex reflection log lambda entries</font>
<div> </div>
</div>
<div class="BodyFragment"><font size="2"><span style="font-size:11pt;">
<div class="PlainText">Hi David.<br>
<br>
Hmmm I am not sure. IIRC the strings are matched as regex. Maybe the $-signs are causing some problems?<br>
<br>
Cheers<br>
Eric<br>
<br>
> On 30. Apr 2019, at 00:31, David Diepenbrock <ddiepenbrock@pjrcorp.com> wrote:<br>
> <br>
> Eric,<br>
> <br>
> Thanks again! That's the pointer I was looking for. However, I'm running into a problem there: when Tamiflex runs the ClassDumper.writeClassesToDisk() the className variable is something like:<br>
> <br>
> javax/swing/plaf/metal/MetalLookAndFeel$$Lambda$6<br>
> <br>
> However, later when processing the reflection log:<br>
> <br>
> javax/swing/plaf/metal/MetalLookAndFeel$$Lambda$6/1338668845<br>
> <br>
> <br>
> Thus, if I add the string "$$Lambda$" to Hasher's instableNames array the map lookup fails when creating the reflection log during ReflLogger.writeLogfileToDisk() specifically in RuntimeLogEntry.replaceByHashedClassName().<br>
> <br>
> I'll keep working at it, but if anyone has ideas about this please send them my way!<br>
> <br>
> -David Diepenbrock<br>
> From: Eric Bodden <eric.bodden@uni-paderborn.de><br>
> Sent: Monday, April 29, 2019 4:26 AM<br>
> To: David Diepenbrock<br>
> Cc: soot-list@cs.mcgill.ca<br>
> Subject: Re: [Soot-list] Tamiflex reflection log lambda entries<br>
> <br>
> Dear David,<br>
> <br>
> I am sorry it seems like I misremembered. I thought I had made this configurable through poa.properties, but looking at the code again I can now see that instead those names are still hard coded, see here:<br>
> <a href="https://github.com/secure-software-engineering/tamiflex/blob/127653932ef1121bb68b0368429687d67363e975/Normalizer/src/de/bodden/tamiflex/normalizer/Hasher.java#L36">
https://github.com/secure-software-engineering/tamiflex/blob/127653932ef1121bb68b0368429687d67363e975/Normalizer/src/de/bodden/tamiflex/normalizer/Hasher.java#L36</a><br>
> <br>
> A short-term solution would thus be to patch this file. An even better solution would be to make this configurable through an option in poa.properties. If you find the time, I’d be happy to accept a pull request along these lines.<br>
> <br>
> Hope this helps!<br>
> <br>
> Cheers<br>
> Eric<br>
> <br>
> > On 27. Apr 2019, at 04:43, David Diepenbrock <ddiepenbrock@pjrcorp.com> wrote:<br>
> > <br>
> > Thanks Eric,<br>
> > I greatly appreciate your response! <br>
> > <br>
> > When you say the config file, are you talking about the play-out-agent's poa.properties file? I have not found any configuration options regarding enabling/disabling via "dontNormalize". Otherwise, I have set the transformations to only enable the first
4 (ClassForName, ClassNewInstance, ConstructorNewInstance and MethodInvoke).<br>
> > <br>
> > I'm reviewing the paper again taking special note to the sections on normalization. I have looked it over briefly and it will certainly be helpful. I will follow up after I've studied the paper carefully.<br>
> > <br>
> > Best,<br>
> > David<br>
> > <br>
> > From: Eric Bodden <eric.bodden@uni-paderborn.de><br>
> > Sent: Thursday, April 25, 2019 9:10 AM<br>
> > To: David Diepenbrock; soot-list@cs.mcgill.ca<br>
> > Subject: Re: [Soot-list] Tamiflex reflection log lambda entries<br>
> > <br>
> > Hi David.<br>
> > <br>
> > Sorry about the late response. In the paper you should find a section on normalizing names. The names of the classes need to be normalized. To configure which names Tamiflex does normalize, you need to edit the appropriate entries in the config file.<br>
> > <br>
> > Does that help?<br>
> > <br>
> > Cheers<br>
> > Eric<br>
> > <br>
> > --<br>
> > Sent from my mobile<br>
> > Von: David Diepenbrock <ddiepenbrock@pjrcorp.com><br>
> > Gesendet: Dienstag, 16. April 2019 23:56<br>
> > An: soot-list@cs.mcgill.ca<br>
> > Betreff: [Soot-list] Tamiflex reflection log lambda entries <br>
> > <br>
> > Hello Soot community, <br>
> > <br>
> > I am having an issue with Soot's reflection log processing and I'm hoping someone might be able to provide some information or guidance. I'm using Tamiflex's play-out-agent to generate a reflection log and that seems to work well. However, when importing
the log Soot is throwing an exception. It seems the problem I'm having is similar to those described in these older postings, to which I did not find any responses:
<br>
> > <a href="https://mailman.cs.mcgill.ca/pipermail/soot-list/2016-October/008587.html">
https://mailman.cs.mcgill.ca/pipermail/soot-list/2016-October/008587.html</a> <br>
> > <a href="https://mailman.cs.mcgill.ca/pipermail/soot-list/2016-March/008418.html">
https://mailman.cs.mcgill.ca/pipermail/soot-list/2016-March/008418.html</a> <br>
> > <br>
> > The specific issue is that lambda usage results in a Constructor.newInstance call such as this being added to the reflection log:
<br>
> > <br>
> > Constructor.newInstance;<java.awt.GraphicsEnvironment$$Lambda$1.1480010240: void <init>()>;java.lang.invoke.InnerClassLambdaMetafactory.buildCallSite;215;isAccessible=true;
<br>
> > <br>
> > Here is an excerpt of the exception during the Soot processing when it attempts to parse the above line of the reflection log (which is the first such Lambda entry).
<br>
> > <br>
> > java.lang.RuntimeException: Unknown method for signature: <java.awt.GraphicsEnvironment$$Lambda$1.1480010240: void <init>()>
<br>
> > at soot.jimple.toolkits.reflection.ReflectionTraceInfo.<init>(ReflectionTraceInfo.java:113) ~[soot-3.1.0.jar:?]
<br>
> > at soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder$TraceBasedReflectionModel.<init>(OnFlyCallGraphBuilder.java:1038) ~[soot-3.1.0.jar:?]
<br>
> > at soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder$TraceBasedReflectionModel.<init>(OnFlyCallGraphBuilder.java:1025) ~[soot-3.1.0.jar:?]
<br>
> > at soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.<init>(OnFlyCallGraphBuilder.java:224) ~[soot-3.1.0.jar:?]
<br>
> > at soot.jimple.toolkits.callgraph.OnFlyCallGraphBuilder.<init>(OnFlyCallGraphBuilder.java:230) ~[soot-3.1.0.jar:?]
<br>
> > at soot.jimple.spark.solver.OnFlyCallGraph.<init>(OnFlyCallGraph.java:75) ~[soot-3.1.0.jar:?]
<br>
> > at soot.jimple.spark.builder.ContextInsensitiveBuilder.setup(ContextInsensitiveBuilder.java:81) ~[soot-3.1.0.jar:?]
<br>
> > at soot.jimple.spark.SparkTransformer.internalTransform(SparkTransformer.java:93) ~[soot-3.1.0.jar:?]
<br>
> > at soot.SceneTransformer.transform(SceneTransformer.java:39) ~[soot-3.1.0.jar:?]
<br>
> > ... <br>
> > <br>
> > If I remove all of the lines from the log containing "$$Lambda$" then soot is able to process the reflection log and I do find that the callgraph has edges from reflection that otherwise were missing. In other words, the reflection log is certainly being
processed and doing what I was hoping/expecting. <br>
> > <br>
> > As noted in the above exception trace, I'm currently using the soot release version 3.1.0. I have also tried with version 3.3.0 with the same results.
<br>
> > <br>
> > I'm wondering if there is some option or setting that I am might be missing, or perhaps there is a general lack of support for lambda (and perhaps other InvokeDynamic related reflection)?
<br>
> > <br>
> > Thanks in advance for any information that can be shared on this topic. <br>
> > <br>
> > Best, <br>
> > David <br>
> > _______________________________________________ <br>
> > Soot-list mailing list <br>
> > Soot-list@CS.McGill.CA <br>
> > <a href="https://mailman.CS.McGill.CA/mailman/listinfo/soot-list">https://mailman.CS.McGill.CA/mailman/listinfo/soot-list</a>
<br>
> <br>
<br>
</div>
</span></font></div>
</body>
</html>