<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Helvetica;
panose-1:2 11 6 4 2 2 2 2 2 4;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";
color:black;}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
p.MsoAcetate, li.MsoAcetate, div.MsoAcetate
{mso-style-priority:99;
mso-style-link:"Sprechblasentext Zchn";
margin:0cm;
margin-bottom:.0001pt;
font-size:8.0pt;
font-family:"Tahoma","sans-serif";
color:black;}
span.E-MailFormatvorlage17
{mso-style-type:personal;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.E-MailFormatvorlage18
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
span.SprechblasentextZchn
{mso-style-name:"Sprechblasentext Zchn";
mso-style-priority:99;
mso-style-link:Sprechblasentext;
font-family:"Tahoma","sans-serif";
color:black;}
.MsoChpDefault
{mso-style-type:export-only;
font-size:10.0pt;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body bgcolor=white lang=DE link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Simone,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>If you just need a list of callback classes, that’s sufficient. In FlowDroid, we, however want a precise callgraph. We, for instance, model that button clicks can only happen when the parent activity is actually running. This allows us to reduce false positives. On the other hand, it means that we always need to know whether a certain call that registers a new callback is reachable from the entry points (or already-known callbacks) of a certain component. Otherwise, we wouldn’t have that mapping between component and callback.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Actually, I’m thinking whether we should provide something similar to what you did as an alternative configuration option. If people are ok with a few false positives, but need something that is fast and consumes only little memory, such a feature would make a lot of sense.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Best regards,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> Steven<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext'>Von:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif";color:windowtext'> Simone Aonzo [mailto:simone.aonzo@gmail.com] <br><b>Gesendet:</b> Donnerstag, 7. April 2016 17:51<br><b>An:</b> Steven Arzt; 'Enzo Lucky'; 'soot-list@CS.McGill.CA'<br><b>Betreff:</b> Re: AW: [Soot-list] Call Graph Generation Performance<o:p></o:p></span></p></div></div><p class=MsoNormal><o:p> </o:p></p><p class=MsoNormal style='margin-bottom:12.0pt'>Hi Steven,<br><br>thanks for the suggestion about the the Exceptions.<br><br>I launch the vm with the parameter:<br>-Xmx 8192m -Xms4096m<br><br>but despite other optimization options the search for the callback needed too much time for my purposes. So I wrote this method:<br><br>protected void internalTransform(String phaseName, @SuppressWarnings("rawtypes") Map options) {<br> System.out.println("--Callback analysis starts--");<br> Iterator<SootClass> classes = Scene.v().getClasses().iterator();<br> while (classes.hasNext()) {<br> SootClass sootClass = classes.next();<br> if (startWithApi(sootClass.toString())) {<br> continue; // this class belong to api<br> }<br> Set<SootClass> allIfaces = collectAllInterfaces(sootClass);<br> for (SootClass iface : allIfaces) {<br> if (androidCallbackClasses.contains(iface.toString())) {<br> List<SootClass> superclasses = getAllSuperclasses(sootClass);<br> classesWithCallbacks.add(new ClassWithCallback<>(sootClass, iface));<br> }<br> }<br> }<br> System.out.println("--Callback analysis ends--");<br> }<br><br>because I don't need to add the callbacks to the dummy method.<br><br><br><o:p></o:p></p><div><p class=MsoNormal>On 07/04/2016 14:40, Steven Arzt wrote:<o:p></o:p></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi all,</span><o:p></o:p></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>The access path length does not influence the callgraph. It is only important for the data flow tracker. Inside the callgraph, there are no access paths. The same holds for enabling or disabling static field tracking, or flow-sensitivity in alias tracking. Exceptions do matter, because when you disable exceptions, the callgraph does not need to consider control flows via uncaught exceptions.</span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>@Simone: How much Java heap space did you allocate? Note that you need to explicitly increase the heap using the –Xmx JVM parameter, e.g., -Xmx4g for 4 GB.</span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Yes, it is indeed the size and complexity of the classes.dex file that matters. The remainder of the APK file has no influence on callgraph construction time.</span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Enabling or disabling callback handling makes a great difference. FlowDroid performs iterative callback analysis. It first takes all entry points that are visible from the manifest file. It then scans for registrations of new callbacks such as button click handlers. If one is found, this callback gets added to the dummy main method and the process is restarted. We need to start over, because we need a new interim callgraph from which we can then again scan for new registrations. In the future, we will have to see whether and how we can avoid the recomputation of all the interim callgraphs, but that’s future work. If you are interested, you might want to look into how we can dynamically extend the interim callgraphs. </span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Best regards,</span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> Steven</span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> </span><o:p></o:p></p><div><div style='border:none;border-top:solid #B5C4DF 1.0pt;padding:3.0pt 0cm 0cm 0cm'><p class=MsoNormal><b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>Von:</span></b><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> <a href="mailto:soot-list-bounces@CS.McGill.CA">soot-list-bounces@CS.McGill.CA</a> [<a href="mailto:soot-list-bounces@CS.McGill.CA">mailto:soot-list-bounces@CS.McGill.CA</a>] <b>Im Auftrag von </b>Enzo Lucky<br><b>Gesendet:</b> Donnerstag, 7. April 2016 12:17<br><b>An:</b> Simone Aonzo; <a href="mailto:soot-list@CS.McGill.CA">soot-list@CS.McGill.CA</a><br><b>Betreff:</b> Re: [Soot-list] Call Graph Generation Performance</span><o:p></o:p></p></div></div><p class=MsoNormal> <o:p></o:p></p><div><div id="yui_3_16_0_ym19_1_1460023453514_3978"><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>Hi Simone,</span><o:p></o:p></p></div><div id="yui_3_16_0_ym19_1_1460023453514_3976"><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'> </span><o:p></o:p></p></div><div id="yui_3_16_0_ym19_1_1460023453514_3977"><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>I am aware of the options (e.g. the access path length) you use, my understanding is using these options will make Soot generate a non-precise call graph. I wanted to know if there are other options to improve performance that wouldn't require sacrificing the precision of the call graph.</span><o:p></o:p></p></div><div id="yui_3_16_0_ym19_1_1460023453514_4348"><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'> </span><o:p></o:p></p></div><div id="yui_3_16_0_ym19_1_1460023453514_4349"><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>Thanks,</span><o:p></o:p></p></div><div id="yui_3_16_0_ym19_1_1460023453514_4350"><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>Enzo</span><o:p></o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt;background:white'><span style='font-family:"Helvetica","sans-serif"'> </span><o:p></o:p></p></div><div><div><div><div><p class=MsoNormal style='background:white'><span style='font-size:10.0pt;font-family:"Arial","sans-serif"'>On Thursday, April 7, 2016 8:48 AM, Simone Aonzo <<a href="mailto:simone.aonzo@gmail.com">simone.aonzo@gmail.com</a>> wrote:</span><o:p></o:p></p></div><p class=MsoNormal style='margin-bottom:12.0pt;background:white'><span style='font-family:"Helvetica","sans-serif"'> </span><o:p></o:p></p><div><div id=yiv5168438696><div><p class=MsoNormal style='margin-bottom:12.0pt;background:white'><span style='font-family:"Helvetica","sans-serif"'><br>Yes, I use FlowDroid.<br>The size of the apk is irrilevant, what matters is the size of the classes.dex<br><br>With my configuration running on an Intel(R) Core(TM) i7-4810MQ CPU @ 2.80GHz:<br>small, 2,3MB, 30 sec<br>medium, 6,8MB, 1 minute<br>big, 11MB (whatsapp), 3 minutes</span><o:p></o:p></p></div><div><p class=MsoNormal style='margin-bottom:12.0pt;background:white'><span style='font-family:"Helvetica","sans-serif"'>With the default configuration I've never seen it halting [but I can't say that it never halts, dear Turing :)] with the medium and the big dex, also after 7 hours.<br><br>Keep in touch!<br>Simone<br><br><br></span><o:p></o:p></p><div id=yiv5168438696yqt99005><div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>On 06/04/2016 19:11, Denis Bogdanas wrote:</span><o:p></o:p></p></div><blockquote style='margin-top:5.0pt;margin-bottom:5.0pt'><div><div><div><div><div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>Interesting,</span><o:p></o:p></p></div><p class=MsoNormal style='margin-bottom:12.0pt;background:white'><span style='font-family:"Helvetica","sans-serif"'>I'm assuming you use FlowDroid. I'm also looking into good options for performance.</span><o:p></o:p></p></div><p class=MsoNormal style='margin-bottom:12.0pt;background:white'><span style='font-family:"Helvetica","sans-serif"'>What size is the apk and how much faster are these settings compared to defaults?</span><o:p></o:p></p></div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>thanks,</span><o:p></o:p></p></div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>Denis</span><o:p></o:p></p></div><div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'> </span><o:p></o:p></p><div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>On 6 April 2016 at 06:05, Simone Aonzo <<a href="mailto:simone.aonzo@gmail.com" target="_blank">simone.aonzo@gmail.com</a>> wrote:</span><o:p></o:p></p><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>This is the way I get a call graph in a reasonable time... if someone<br>has better ideas: share!<br><br> soot.G.reset();<br> SetupApplication setupApplication = new<br>SetupApplication(androidJar.toString(), apk.toString());<br> setupApplication.getConfig().setFlowSensitiveAliasing(false);<br>// --aliasflowins<br> setupApplication.getConfig().setAccessPathLength(1); // --aplength n<br> setupApplication.getConfig().setEnableStaticFieldTracking(false);<br>// --nostatic<br> setupApplication.getConfig().setEnableCallbacks(false);// --nocallbacks<br> setupApplication.getConfig().setPathBuilder(DefaultPathBuilderFactory.PathBuilder.ContextSensitive);//<br>--pathalgo<br> setupApplication.getConfig().setComputeResultPaths(false); // --nopaths<br> setupApplication.getConfig().setEnableExceptionTracking(false);<br>// --noexceptions<br> setupApplication.calculateSourcesSinksEntrypoints(sourcesAndSinks.toString());<br><br> Options.v().set_src_prec(Options.src_prec_apk);<br> Options.v().set_soot_classpath("/usr/lib/jvm/java-8-oracle/jre/lib/rt.jar:/usr/lib/jvm/java-8-oracle/jre/lib/jce.jar:.");<br> Options.v().set_process_dir(Collections.singletonList(apk.toString()));<br> Options.v().set_android_jars(androidJar.toString());<br><br> List<String> excludeList = new LinkedList<String>();<br> excludeList.add("java.");<br> excludeList.add("sun.misc.");<br> excludeList.add("android.");<br> excludeList.add("org.apache.");<br> excludeList.add("soot.");<br> excludeList.add("javax.servlet.");<br><br> Options.v().set_exclude(excludeList);<br> Options.v().set_no_bodies_for_excluded(true);<br> Options.v().set_output_format(Options.output_format_none);<br> Options.v().set_allow_phantom_refs(true);<br> Options.v().set_whole_program(true);<br> Options.v().setPhaseOption("cg.spark", "on");<br> Options.v().setPhaseOption("cg", "trim-clinit:false");<br> SootMethod dummyMain =<br>setupApplication.getEntryPointCreator().createDummyMain();<br> Options.v().set_main_class(dummyMain.getSignature());<br> Scene.v().setEntryPoints(Collections.singletonList(dummyMain));<br> PackManager.v().runPacks();<br> CallGraph callGraph = Scene.v().getCallGraph();<br><br>2016-04-04 12:37 GMT+02:00 Enzo Lucky <<a href="mailto:luckenzo24@yahoo.com" target="_blank">luckenzo24@yahoo.com</a>>:<br>> Hi,<br>><br>> I am using Soot to generate call graph of some Android apps, and I find Soot<br>> take more than a day to generate the call graph for some apps. Can I do<br>> anything other than the suggestions in the FlowDroid homepage<br>> (<a href="https://github.com/secure-software-engineering/soot-infoflow-android/wiki#improving-performance" target="_blank">https://github.com/secure-software-engineering/soot-infoflow-android/wiki#improving-performance</a>)?<br>> I want a precise call graph and hence, I didn't use any of the options.<br>> Also, is the time normal or am I doing something wrong? Please find attached<br>> my dummy main class.<br>><br>> Thanks,<br>> Enzo<br>><br>> _______________________________________________<br>> Soot-list mailing list<br>> <a href="mailto:Soot-list@CS.McGill.CA" target="_blank">Soot-list@CS.McGill.CA</a><br>> <a href="https://mailman.cs.mcgill.ca/mailman/listinfo/soot-list" target="_blank">https://mailman.CS.McGill.CA/mailman/listinfo/soot-list</a><br>><br>_______________________________________________<br>Soot-list mailing list<br><a href="mailto:Soot-list@CS.McGill.CA" target="_blank">Soot-list@CS.McGill.CA</a><br><a href="https://mailman.cs.mcgill.ca/mailman/listinfo/soot-list" target="_blank">https://mailman.CS.McGill.CA/mailman/listinfo/soot-list</a></span><o:p></o:p></p></div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'><br><br clear=all><br>-- </span><o:p></o:p></p><div><div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>Denis</span><o:p></o:p></p></div></div></div></blockquote></div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'> </span><o:p></o:p></p></div></div><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'> </span><o:p></o:p></p><div id=yqt62553><p class=MsoNormal style='background:white'><span style='font-family:"Helvetica","sans-serif"'>_______________________________________________<br>Soot-list mailing list<br><a href="mailto:Soot-list@CS.McGill.CA">Soot-list@CS.McGill.CA</a><br><a href="https://mailman.cs.mcgill.ca/mailman/listinfo/soot-list" target="_blank">https://mailman.CS.McGill.CA/mailman/listinfo/soot-list</a></span><o:p></o:p></p></div><p class=MsoNormal style='margin-bottom:12.0pt;background:white'><span style='font-family:"Helvetica","sans-serif"'> </span><o:p></o:p></p></div></div></div></div></div></blockquote><p class=MsoNormal><o:p> </o:p></p></div></body></html>