<html xmlns:v="urn:schemas-microsoft-com:vml" xmlns:o="urn:schemas-microsoft-com:office:office" xmlns:w="urn:schemas-microsoft-com:office:word" xmlns:m="http://schemas.microsoft.com/office/2004/12/omml" xmlns="http://www.w3.org/TR/REC-html40"><head><meta http-equiv=Content-Type content="text/html; charset=utf-8"><meta name=Generator content="Microsoft Word 14 (filtered medium)"><style><!--
/* Font Definitions */
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Wingdings;
panose-1:5 0 0 0 0 0 0 0 0 0;}
@font-face
{font-family:Calibri;
panose-1:2 15 5 2 2 2 4 3 2 4;}
@font-face
{font-family:Tahoma;
panose-1:2 11 6 4 3 5 4 4 2 4;}
/* Style Definitions */
p.MsoNormal, li.MsoNormal, div.MsoNormal
{margin:0cm;
margin-bottom:.0001pt;
font-size:12.0pt;
font-family:"Times New Roman","serif";}
a:link, span.MsoHyperlink
{mso-style-priority:99;
color:blue;
text-decoration:underline;}
a:visited, span.MsoHyperlinkFollowed
{mso-style-priority:99;
color:purple;
text-decoration:underline;}
span.E-MailFormatvorlage17
{mso-style-type:personal-reply;
font-family:"Calibri","sans-serif";
color:#1F497D;}
.MsoChpDefault
{mso-style-type:export-only;
font-family:"Calibri","sans-serif";
mso-fareast-language:EN-US;}
@page WordSection1
{size:612.0pt 792.0pt;
margin:70.85pt 70.85pt 2.0cm 70.85pt;}
div.WordSection1
{page:WordSection1;}
/* List Definitions */
@list l0
{mso-list-id:82187902;
mso-list-template-ids:134151590;}
@list l0:level1
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:36.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Symbol;}
@list l0:level2
{mso-level-number-format:bullet;
mso-level-text:o;
mso-level-tab-stop:72.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:"Courier New";
mso-bidi-font-family:"Times New Roman";}
@list l0:level3
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:108.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level4
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:144.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level5
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:180.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level6
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:216.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level7
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:252.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level8
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:288.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
@list l0:level9
{mso-level-number-format:bullet;
mso-level-text:;
mso-level-tab-stop:324.0pt;
mso-level-number-position:left;
text-indent:-18.0pt;
mso-ansi-font-size:10.0pt;
font-family:Wingdings;}
ol
{margin-bottom:0cm;}
ul
{margin-bottom:0cm;}
--></style><!--[if gte mso 9]><xml>
<o:shapedefaults v:ext="edit" spidmax="1026" />
</xml><![endif]--><!--[if gte mso 9]><xml>
<o:shapelayout v:ext="edit">
<o:idmap v:ext="edit" data="1" />
</o:shapelayout></xml><![endif]--></head><body lang=DE link=blue vlink=purple><div class=WordSection1><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Hi Jason,<o:p></o:p></span></p><p class=MsoNormal><span style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>You put a directory “android_source_dir“ as the Soot process directory. This means that all classes in there will be read in, including all associated method bodies. What exactly are the contents of this directory? Is this the full implementation of the Android OS? If so, you will most likely need a lot of memory. Why don’t you just put that directory on the Soot classpath alone and leave it out from the process directory? As long as you don’t want to instrument these classes, it’s fine to just have them “in the back” so that Soot can load those classes which are actually required, instead of pro-actively loading them all.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Secondly, I see that you use a non-static method as your entry point. This will not give you a complete callgraph. SPARK, the Soot pointer-analysis and callgraph framework, works by propagating allocation nodes. If you jump into an instance method, SPARK will never see an allocation site for the “this” local inside this method and will thus miss out on all edges from this local.<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'>Best regards,<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'> Steven<o:p></o:p></span></p><p class=MsoNormal><span lang=EN-US style='font-size:11.0pt;font-family:"Calibri","sans-serif";color:#1F497D'><o:p> </o:p></span></p><p class=MsoNormal><b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>Von:</span></b><span lang=EN-US style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'> soot-list-bounces@CS.McGill.CA [mailto:soot-list-bounces@CS.McGill.CA] <b>Im Auftrag von </b>Jason Ott<br><b>Gesendet:</b> Donnerstag, 2. A</span><span style='font-size:10.0pt;font-family:"Tahoma","sans-serif"'>pril 2015 20:15<br><b>An:</b> soot-list@CS.McGill.CA<br><b>Betreff:</b> [Soot-list] Soot/Spark GC overhead limit exceeded and the Android Framework<o:p></o:p></span></p><p class=MsoNormal><o:p> </o:p></p><div><div><p class=MsoNormal>So I'm trying to run soot with spark on the Android Framework using a custom entry point.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Here is my the code that runs soot:<o:p></o:p></p></div><div><p class=MsoNormal> <o:p></o:p></p></div><div><p class=MsoNormal> public class Main {<o:p></o:p></p></div><div><p class=MsoNormal> public static void main(String[] args) {<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> String entryPointClass = args[args.length-2];<o:p></o:p></p></div><div><p class=MsoNormal> String entryPointMethod = args[args.length-1];<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> System.out.println(entryPointClass + "." + entryPointMethod);<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> args = Arrays.copyOf(args, args.length-2);<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> long start = System.currentTimeMillis();<o:p></o:p></p></div><div><p class=MsoNormal> Options.v().parse(args);<o:p></o:p></p></div><div><p class=MsoNormal> SootClass c = Scene.v().forceResolve(entryPointClass, SootClass.BODIES);<o:p></o:p></p></div><div><p class=MsoNormal> c.setApplicationClass();<o:p></o:p></p></div><div><p class=MsoNormal> Scene.v().loadNecessaryClasses();<o:p></o:p></p></div><div><p class=MsoNormal> SootMethod method = c.getMethodByName(entryPointMethod);<o:p></o:p></p></div><div><p class=MsoNormal> List entryPoints = new ArrayList();<o:p></o:p></p></div><div><p class=MsoNormal> entryPoints.add(method);<o:p></o:p></p></div><div><p class=MsoNormal> Scene.v().setEntryPoints(entryPoints);<o:p></o:p></p></div><div><p class=MsoNormal> PackManager.v().runPacks();<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> soot.jimple.toolkits.callgraph.CallGraph callgraph = Scene.v().getCallGraph();<o:p></o:p></p></div><div><p class=MsoNormal> System.out.println("[TestSpark] Call graph size " + callgraph.size());<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal> System.out.println(String.format("It took: %d seconds to generate a callgraph for: %s",<o:p></o:p></p></div><div><p class=MsoNormal> TimeUnit.MILLISECONDS.toSeconds(System.currentTimeMillis() - start),<o:p></o:p></p></div><div><p class=MsoNormal> entryPointClass +"."+ entryPointMethod));<o:p></o:p></p></div><div><p class=MsoNormal> }<o:p></o:p></p></div><div><p class=MsoNormal>}<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>I would run this as follows:<o:p></o:p></p></div><div><p class=MsoNormal>java -Xmx:1024mb -jar mySpark.jar -p cg.spark on -process-dir /Users/jason/Desktop/android/pscout/android_source_dir/ -cp /Users/jason/Desktop/android/pscout/android_source_dir:. -allow-phantom-refs android.content.ClipboardManager hasText<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>The last two args end up being the api class and the api method that I want to start the graph at. <o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>When I run this jar I get a "java.lang.OutOfMemoryError: GC overhead limit exceeded" error. I've given the JVM as mucha s 8gb and it still exhausts memory.<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Now I must admit that to even get this to work I had to tweak the Soot source code a bit. I had to alter the following:<o:p></o:p></p></div><div><ul type=disc><li class=MsoNormal style='mso-margin-top-alt:auto;mso-margin-bottom-alt:auto;mso-list:l0 level1 lfo1'>Line 99 in soot/src//soot/asm/SootClassBuilder.java will not throw an exception with my code. The line now prints an error and moves on. As I was reading the code and reading other examples, this is tolerable because Soot still finds the methods in those classes (presumably, as it doesn't throw any errors about not finding methods)<o:p></o:p></li></ul><div><p class=MsoNormal>Can soot handle the entire android framework?<o:p></o:p></p></div></div><div><p class=MsoNormal>Is my method and approach proper?<o:p></o:p></p></div><div><p class=MsoNormal>Are there any indications as to why I'm running into a GC Memory error and how to fix it?<o:p></o:p></p></div><div><p class=MsoNormal>Is my change to soot source code the root of this problem?<o:p></o:p></p></div><div><p class=MsoNormal><o:p> </o:p></p></div><div><p class=MsoNormal>Thank you for any assistance you may provide.<o:p></o:p></p></div></div></div></body></html>