Dalvik bytecode represents 'null' as 0 which may lead to problems
in phi instructions. This variant of TypeInference fixes these
problems by
- tracking whether an SSA value is constant zero and
- ignoring constant zeros in the transfer function of phi instructions
when meeting with non-primitive types
Previously, the various Eclipse projects' Java configurations used
mixtures of 1.6, 1.7, and 1.8. Many were internally inconsistent,
such as requiring 1.7 in "MANIFEST.MF" but 1.6 in the Eclipse JDT
build preferences. The Travis-CI configuration tests against both 1.7
and 1.8, but does not test against 1.6.
Across all projects, the most common version was 1.7. So I'm going to
assume that 1.7 is the intended build target. This commit makes 1.7
the selected version nearly everywhere.
"com.ibm.wala.core.testdata" is the one exception. This specific
project uses a few features only found in 1.8, such as lambda
expressions. Previously, "com.ibm.wala.core.testdata" used 1.7 in
some aspects of its configuration but 1.8 in others. Now it
consistently targets 1.8. I wish this one project didn't need to be
inconsistent with the rest of WALA, but at least now it's consistent
with itself.
(Personally, I'd be happy to target 1.8 only. But my impression
across all of these configuration files is that the WALA developers
still want to be compatible with 1.7. If that is no longer a
requirement, let me know and I will adjust these changes accordingly
to target 1.8 only.)
This change eliminates 11 "There is no 'jre.compilation.profile' build
entry and the project has Java compliance preferences set" warnings
and 13 "The JRE container on the classpath is not a perfect match to
the 'JavaSE-1.7' execution environment" warnings. However, it also
adds 450 "Redundant specification of type arguments <...>" warnings
and 17 "Resource '...' should be managed by try-with-resource"
warnings. So this seems like a net step backward in my wish to reduce
WALA warnings. However, those new warnings concern Java 1.7 language
features that we were not previously using to good effect in projects
that targeted 1.6. If we all agree that we can now target 1.7
instead, then we can use these helpful features as the newly-added
warnings suggest. So I call that a step in the right direction.
When Maven generates these "*/target/antrun/build-main.xml" Any build
scripts, it does not include any DTD or XML Schema declarations.
Eclipse's XML validator warns about the lack of grammar constraints.
The warning is sensible, but we are not in a position to do anything
about it. Better, therefore, to suppress these warnings so that we
can more-clearly see warnings we *can* address.
Some of these might have proper DTDs or XML Schema definitions
floating around somewhere that we could use. Presumably many do not.
Rather than hand-craft such definitions myself, I'm just giving each a
minimal stub DOCTYPE declaration. That's enough to satisfy Eclipse's
XML validator, which otherwise complains that these files lack grammar
constraints.
I think the "target/p2artifacts.xml" and "target/p2content.xml" files
are generated by Maven. They are well-formed XML but Eclipse's XML
validator legitimately warns that they lack grammar constraints.
Since we're not maintaining the tool that creates these files, we are
not in a position to do anything about that. Therefore, we may as
well exclude these from validation entirely. That way we can
more-clearly recognize warnings that we *can* do something about.
Ant "build.xml" files don't have a standard DTD or XML Schema; the
contents are simply too flexible for that. But we can at least
give each a stub DOCTYPE declaration. That's enough to satisfy
Eclipse's XML validator, which otherwise complains that these files
lack grammar constraints.
Eclipse validation warns about invalid HTML content in all
Maven-generated "target/site/dependency-convergence.html" files. The
warnings are legitimate: these HTML files are indeed invalid.
However, we don't maintain the tool that generates these files, so we
are not in a position to fix them. Better, therefore, to suppress
these warnings so that we can notice and fix other problems over which
we do have control.
If a client violates these restrictions, I prefer that their code fail
at compile time instead of run time. Changing a few key types from
`Class` to `Class<? extends AbstractAndroidModel>` gives us precisely
the static enforcement we need and lets us remove an
`AbstractAndroidModel.class.isAssignableFrom` run-time check.
However, this does change the public API of `AndroidEntryPointManager`
in two ways. The `getModelBehavior` and `setModelBehavior` methods now
respectively accept and return `Class<? extends AbstractAndroidModel>`
instead of `Class`. Is tightening up a public API in this manner
considered OK?
This resolves one Eclipse "'...' build entry is missing" warning.
Also update the project configuration to treat this warning as an
error. This should discourage commits that create new instances of
this sort of problem in the future.
This resolves one "'...' is not a source folder" Eclipse warning.
Also update the project configuration to treat this warning as an
error. This should discourage commits that create new instances of
this sort of problem in the future.
In general, the WALA code base is not really ready for nullness
checking. It would be nice if we got there some day, but I'm not
planning to take that on now or any time soon. Until then, it's not
useful to warn about missing @NonNullByDefault declarations on WALA
packages.
See also older commit 7b6811b.
Eclipse Mars Service Release 2 finds 45 potential null pointer accesses
across WALA's various Eclipse projects. Eclipse ignores these by
default, but any individual user may have changed their personal Eclipse
configuration to treat them as warnings or errors. Thus, some people
will find that the code builds while others find that it fails. Better
to explicitly use a known-good configuration.
In the long run someone should inspect these cases one-by-one and fix
them where appropriate. But that is probably better managed as part of a
larger effort to tidy up nulls in WALA. I'm not planning to take that on
now or any time soon, though, so this is a better setup for now.
This way, the rest of the outer loop (including the jump to the
beginning) is also reached in the control-flow which means that
the outer loop is modelled as a proper loop
a) serializable added for use by Android services
b) test classes refactored to allow Android variants to use JUnit 3
2) shrike instrumentation now uses java.lang.instrument
a) refactoring
b) online variants of call graph tracing
now, for me, code works using e44 with maven
dalvik tests refactored for mobile version with android dev tools
IDE tests Eclipse metadata fixed to make e44 work for me
new android entrypoint to fix failure in new droidbench tests
com.ibm.wala.cast.js.rhino.test/harness-src/com/ibm/wala/cast/js/test/TestPrototypeCallGraphShapeRhino.java
com.ibm.wala.cast.js.test/harness-src/com/ibm/wala/cast/js/test/TestPrototypeCallGraphShape.java
com.ibm.wala.cast.js.test.data/examples-src/pages/prototype.html
work (not yet finished) on fixes to property accesses for JavaScript:
com.ibm.wala.cast/source/java/com/ibm/wala/cast/ipa/callgraph/AstSSAPropagationCallGraphBuilder.java
com.ibm.wala.cast.java/src/com/ibm/wala/cast/java/ipa/callgraph/AstJavaSSAPropagationCallGraphBuilder.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/ipa/callgraph/JSSSAPropagationCallGraphBuilder.java
currently unused tests to remind me to fix bugs:
com.ibm.wala.cast.js.test/harness-src/com/ibm/wala/cast/js/test/TestSimpleCallGraphShape.java
com.ibm.wala.cast.js.test.data/examples-src/tests/loops.js
com.ibm.wala.cast.js.test.data/examples-src/tests/primitive_strings.js
fixes to exception handler code generation in JavaScript:
com.ibm.wala.cast.js.rhino/source/com/ibm/wala/cast/js/translator/RhinoToAstTranslator.java
com.ibm.wala.cast.js.test.data/examples-src/tests/try.js
com.ibm.wala.cast.js.test/harness-src/com/ibm/wala/cast/js/test/TestSimpleCallGraphShape.java
fixes to make the system build on both juno and luna
com.ibm.wala.cast.js.test.data/pom.xml
pom.xml
targets/e42/e42.target
targets/e44/e44.target
targets/pom.xml
com.ibm.wala.core.tests/META-INF/MANIFEST.MF
com.ibm.wala.dalvik.test/META-INF/MANIFEST.MF
com.ibm.wala.ide.jdt.test/META-INF/MANIFEST.MF
com.ibm.wala.ide.jdt/source/com/ibm/wala/cast/java/translator/jdt/FakeExceptionTypeBinding.java
com.ibm.wala.ide.jdt/source/com/ibm/wala/ide/util/JavaEclipseProjectPath.java
com.ibm.wala.ide.jsdt.tests/META-INF/MANIFEST.MF
com.ibm.wala.ide.jsdt.tests/src/com/ibm/wala/ide/jsdt/tests/AbstractJSProjectScopeTest.java
com.ibm.wala.ide/src/com/ibm/wala/ide/util/EclipseProjectPath.java
com.ibm.wala.ide/src/com/ibm/wala/ide/util/ProgressMonitorDelegate.java
beginnings of "pointer analysis" on top of field-based analysis
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/FlowGraph.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/vertices/PropVertex.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/vertices/RetVertex.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/vertices/VarVertex.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/vertices/VertexFactory.java
com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/PointerAnalysis.java
com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/cfa/ExceptionReturnValueKey.java
fixes for crashes in correlartion tracking
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/ipa/callgraph/correlations/extraction/ClosureExtractor.java
fixes for Dalvik IR generation
com.ibm.wala.core/src/com/ibm/wala/cfg/BytecodeCFG.java
com.ibm.wala.core/src/com/ibm/wala/cfg/ShrikeCFG.java
com.ibm.wala.core/src/com/ibm/wala/ssa/SSACFG.java
com.ibm.wala.dalvik.test/source/com/ibm/wala/dalvik/drivers/APKCallGraphDriver.java
com.ibm.wala.dalvik.test/source/com/ibm/wala/dalvik/test/callGraph/JVMLDalvikComparison.java
com.ibm.wala.dalvik/src/com/ibm/wala/dalvik/classLoader/DexCFG.java
com.ibm.wala.dalvik/src/com/ibm/wala/dalvik/dex/instructions/UnaryOperation.java
com.ibm.wala.dalvik/src/com/ibm/wala/dalvik/ssa/AbstractIntRegisterMachine.java
com.ibm.wala.dalvik/src/com/ibm/wala/dalvik/ssa/DexSSABuilder.java
fixes to stack map generation when instrumenting for Java 7
com.ibm.wala.shrike/src/com/ibm/wala/shrike/cg/DynamicCallGraph.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/ConstantInstruction.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/analysis/Analyzer.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/analysis/ClassHierarchy.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/analysis/Verifier.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/shrikeCT/ClassInstrumenter.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeCT/StackMapConstants.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeCT/StackMapTableReader.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeCT/StackMapTableWriter.java
Now there are four structural models:
* SequentialAndroidModel: No loops
* SingleStartAndroidModel: User Interaction on a single component
* LoopAndroidModel: Stuff goes into background and comes back
* LoopKillAndroidModel: Restart of components due to low memory
Code oftain sets the action of an Intent after it's constructor. Until
now a call to such a setter caused the Intent to become "unbound"
(conservative).
This approch allows setting the target once for each Intent - only on
the second call the Intent gets unbound.
This new variant could be dangerous: Setting the target in a branch of
execution may be invalid. This should be detected - no guarantees so!
Methods in question are:
* Intent.setAction
* Intent.setComponent
* Intent.setClass
* Intent.setClassName
Before the Intent would only have been set by calling .attach (if
doBootSequence is enabled). Attach is only called when the modell is
filled with instructions.
This new variant calls setIntent when creating the wrapper for the model
(getMethodAs). This is much better!
* Create IntentContext even if no info available.
This is necessary to also track the start of unknown targets.
* Invalidate the target of an Inten upon a call to Intent.setAction
or Intent.fillIn
Additionally tidied up the classes a bit.
The SystemServiceModel creates and returns a new Instance of the
requested Service (if known).
TODO: We should use a single "global" instance per service instead.
Building the CFG with a SSAGotoInstuction was buggy: Oftain the wrong
jump-target was selected. This has bin fixed.
Additionally InducedCFG now automaticly breaks the basic-block at the
jump-target.
Jumping to Phi-Instructions however is still unsupported (as they are
not part of the cfg-instructions)
There was (and is) the posibility of an endless-loop when looking up the
targets of Intents. Added an evil hack to at least prevent the most
obvious one.
The Analysis has problems with the IntentSender class: Invalid DefUse an
an Init-Call.
The Intent-Starters, that use IntentSender have been made optional.
However disabling them does not help.
Currently the only solution to this problem is adding IntentSender to
the exclusions.txt
Setting the Instantiation-Behavior of Package:
Landroid/support/v4/view
To REUSE causes the following Endless-Recursion in JoDroid:
interproc: computing local killing defintions... done
Building utility edges Exception in thread "main" java.lang.StackOverflowError
at java.util.HashMap.put(HashMap.java:389)
at org.jgrapht.graph.AbstractBaseGraph.addEdge(Unknown Source)
at edu.kit.joana.util.graph.AbstractJoanaGraph.addEdge(AbstractJoanaGraph.java:50)
at edu.kit.joana.wala.core.DependenceGraph.addEdge(DependenceGraph.java:76)
at edu.kit.joana.wala.core.joana.JoanaConverter$UtilityEdgeWalker.discover(JoanaConverter.java:344)
at edu.kit.joana.wala.core.joana.JoanaConverter$UtilityEdgeWalker.discover(JoanaConverter.java:325)
at edu.kit.joana.wala.core.graphs.GraphWalker.dfs(GraphWalker.java:55)
at edu.kit.joana.wala.core.graphs.GraphWalker.dfs(GraphWalker.java:63)
at edu.kit.joana.wala.core.graphs.GraphWalker.dfs(GraphWalker.java:63)
at edu.kit.joana.wala.core.graphs.GraphWalker.dfs(GraphWalker.java:63)
...
As this package contains System-provided android-components it should however be marked REUSE in order to be able to read back values when these types are used.
AndroidPreFlightChecks provides some checkups on the settings before
bulding the Livecycle.
The checks have to be invoked explicitly and issue warnings in the log.
Governed by AndroidEntryPointManager.setDoBootSequence generate some
coarse Android-environment before starting the LiveCycle-Model.
Some action is taken to attach the Android-Context to Components - much
is still missing there though.
Thin context is mainly useful when starting Intents of an external App.
Whenever an Intent is encountered while building the CallGraph a
WALA-Context is generated for it. If a new Android-Component is started
and the Context is sufficient the start-function will call a new type of
model, the MicroModel.
The MicroModel resemples the livecycle of a single Android-Component.
Whenever the start of an intent is encountered the start-function is
replaced by a call to UnknownTargetModel.
UnknownTargetModel will call a restricted android-model (i.e. one that calls only all
Activities). This restricted model is known as MiniModel.
It will also call ExternalModel which does nothing special.
Generate a synthetic AndroidModel in AndroidModelClass.
The model will contain Android EntryPoints in a sorted manner. However
no special handling (loops) are inserted yet.
Intents are not processed at all - thus have to be marked insecure.
Parameters to Androids EntryPoint-Functions may be either marked CREATE
or REUSE. Added these markers and made them availabel through
AndroidEntryPointManager.
Using a SummarizedMethodWithNames instead of a normal one enables human
readable variable names in WALA-Synthetic methods. This should help
when debugging.
Depending on the method used generating an AnalysisScope failed for
Android-Apps. Especially depending on wheater data was used from a
jar-resource or depending on exclusions.txt
CAUTION: Now you have to make sure that the provided android lib actually contains all standard
java classes (e.g. java.lang.Object); WALA will complain and crash if this is not the case
Martin Mohr
Achim D. Brucker
Martin Hecker
Ben Liblit
Manu Sridharan
Julian Dolby
Manu Sridharan
Michael Herzberg
Juergen Graf
Tobias Blaschke