These are all problems that Eclipse can detect, but that it detects no
instances of right now. Treating these as warnings instead of errors
should help prevent us from slipping backward in the future.
Each of these required careful consideration of what the original
developer *intended* as distinguished from what the developer's code
actually *does*. I believe I got each one right, and WALA's
regression tests agree. A second opinion by a core WALA developer
would be welcome, though.
We already have plenty of examples of Serializable classes with this
field, and the vast majority of those fields have generated IDs rather
than "1L". From this I infer that using proper serialVersionUID
fields is considered appropriate WALA coding style.
One such annotation was unnecessary because the thing it was
suppressing no longer happens. Any future unnecessary warning
suppressions of this kind will now be treated as errors.
The other annotations were unnecessary because the corresponding
warnings have been disabled entirely in the Eclipse projects'
configurations. There seems to be no way to tell Eclipse to treat
these as anything other than "info" diagnostics in the future, so
that's how they will remain.
In general, my approach was to try to eliminate each unused parameter
using Eclipse's "Change Method Signature" refactoring. That did not
always succeed: a parameter may be unused in some base class method,
but then be used in subclass's override of that method. In cases
where refactoring to eliminate a parameter failed, I instead annotated
the parameter with '@SuppressWarnings("unused")' to silence the
warning.
Note: this group of changes creates a significant risk of
incompatibility for third-party WALA code. Some removed parameters
change externally-visible APIs. Furthermore, these changes do not
necessarily lead to Java compilation errors. For example, suppose
third-party code subclasses a WALA class or interface, overrides a
method, but does not annotate that method as @Override. Removing a
parameter means that the third-party method no longer overrides. This
can quietly change code behavior without compile-time errors or
warnings. This is exactly why one should use @Override wherever
possible, but we cannot guarantee that third-party WALA users have
done that.
Unnecessary "throws" declarations tend to cascade. If foo() calls
bar() and bar() falsely declares that it might throw IOException, that
often leads a programmer to declare that foo() might throw IOException
as well. Fixing the bar() throws declaration then reveals that we can
fix the foo() throws declaration too. By the time we reach a fixed
point with cleaning these up, we have removed roughly 320 unnecessary
throws declarations.
In a few cases, this cleanup even lets us remove entire "try
... catch" statements where the only thing being caught was an
exception that we now statically know cannot be thrown. Nice!
In Eclipse project configurations, upgrade any future such shenanigans
from warnings to errors. Now that we've fixed this, we don't want it
coming back again.
There is a potential drawback to this change. Conceivably some public
WALA API entry point might have declared that it could throw some
exception merely to reserve the *option* of throwing that exception in
third-party code that subclasses and overrides the API entry point in
question. I have no idea whether this is a significant concern in
practice, though.
Previously we had 227 such warnings. That large number suggests that
the WALA developers consider this to be an acceptable coding style.
If that's so, then it's better to hide these warnings rather than keep
them around as a perpetual distraction.
Previously we had 242 such warnings. That large number suggests that
the WALA developers consider this to be an acceptable coding style.
If that's so, then it's better to hide these warnings rather than keep
them around as a perpetual distraction.
I have *not* upgraded this problem to be treated as an error in the
future. Unfortunately Eclipse uses a single configuration setting for
both unnecessary semicolons and also for empty control-flow statements
like `while (p) ;`. I'm not convinced that it's worth rewriting all
instances of the latter into `while (p) { }`. So this is just going
to stay as a warning for now.
In "com.ibm.wala.cast.test", the "cbuild.sh" script is gone entirely.
Instead, "pom.xml" directs Maven to run "make" directly. We still
have a "Makefile.configuration" file in this project, but this file is
now independent of where WALA is being built and of where Java is
installed.
In "com.ibm.wala.cast", a small "cbuild.sh" remains, to do some
special processing involving Visual Studio variables under Windows.
When not building under Windows, "cbuild.sh" now simply runs "make".
It may well be possible to hoist the special Windows stuff up into
this subproject's "pom.xml", or to change that "pom.xml" to run
"cbuild.sh" only when on Windows, and to run "make" directly
otherwise. I don't know "pom.xml" stuff very well, though. We still
have a "Makefile.configuration" file in this project, but this file is
now independent of where WALA is being built and of where Java is
installed.
The changes I've made in both "Makefile.configuration" files use GNU
make extensions. I assume that's safe because "Makefile.definitions"
already relies on GNU make.
In general, these diagnostics are now errors in projects for which all
such warnings have been fixed. There are three unfixed warnings in
two projects, so this diagnostic remains a warning (not an error) in
those projects.
There are also many places where rwa-types-usage warnings have been
locally suppressed using @SuppressWarnings annotations. I haven't
systematically revisited those to see if any can be fixed properly.
So for those projects this diagnostic must also remain a warning (not
an error), since @SuppressWarnings does not work on things Eclipse is
configured to treat as errors.
Along the way, I also converted many "for (;;)" loops into modern
"for (:)" loops. I didn't systematically look for all opportunities
to do this, though. I merely made this change where I was already
converting raw Iterator uses into modern Iterator<...> uses.
Better use of generics also allowed many casts to become statically
redundant. I have removed all such redundant casts.
Only three raw-types warnings remain after this batch of fixes. All
three involve raw uses of CallGraphBuilder. I've tried to fix these
too, but it quickly snowballs into a cascade of changes that may or
may not eventually reach a statically-type-save fixed point. I may
give these last few problem areas another go in the future. For now,
though, the hundreds of other fixes seem worth keeping even if there
are a few stragglers.
This commit may change some public APIs, but only by making weaker
type signatures stronger by replacing raw types with generic types.
For example, we may change something like "Set" into "Set<String>",
but we're not adding new arguments, changing any
underlying (post-generics-erasure) types, etc.
There are two such diagnostics: one for collection methods and one for
equals(). See
<https://www.eclipse.org/eclipse/news/4.7/jdt.php#unlikely-argument-types>
for more information about these two new diagnostics.
For each of these diagnostics, I've set the severity level to
"warning" in projects that have some instances of the suspicious code,
or to "error" in projects that have no instances of the suspicious
code.
These should mostly be things that we've already decided earlier that
we explicitly don't want to "fix" because they simply disagree with
the WALA project's coding style.
The additional diagnostics are ones that were previously being
ignored, but which we seem to have been ignoring by default rather
than as a conscious choice.
For diagnostics of which we currently have *zero* instances, treat
these as errors rather than merely warnings. The intent is to
permanently lock out future regressions of things we've completely
fixed. In the future, whenever we fix the last instance of a given
warning in a given Eclipse project, we should also promote that
diagnostic to an error to keep things clean into the future.
Previously some of these were accessing such fields through a subclass
of the declaring class. That creates an unnecessary extra inter-class
dependency lower in the type hierarchy than necessary.
Also, suppress this warning in an automated test input where the
indirect static accesses are explicitly intentional.
The method itself accesses only static members. The fact that it is
final means no subclass can ever override this method and use dynamic
dispatch to choose the implementation at run time. So declaring this
method static is (statically) safe.
If a method is private, there's no risk that a subclass elsewhere
might be overriding it and depending on dynamic dispatch to choose the
right implementation. So all of these private methods can safely be
declared static without risk of regression in either WALA code or
unseen third-party code.
The "potentially" qualifier is here because these methods are visible
outside the WALA source tree. These methods may seem OK to be static
based on the code we have here, but we have no way of knowing whether
third-party code expected to be able to subclass and override. I'm
going to play it safe and assume that we want to allow that.
Note that we are still allowing Eclipse warnings about methods that
can *definitely* be declared static; a different configuration option
controls these. For private methods, final methods, and methods in
final classes, if the code seems static-safe based on what we have
here, then that's good enough: we don't need to worry about
third-party overrides.
* Fix warnings about unset javacProjectSettings build entries
Specifically, these are all warnings of the form "The
'javacProjectSettings' build entry should be set when there are project
specific compiler settings".
* Add @Override annotations to all methods that do override
This fixes 287 Eclipse code style warnings.
* Cannot add @Override annotations here, so suppress warnings instead
We should be able to add these @Override annotations in the future,
one Eclipse Mars and earlier are no longer supported. For now,
though, they have to go away in order to be compatible with older
Eclipse releases.
Specifically, these are all warnings of the form "The
'javacProjectSettings' build entry should be set when there are project
specific compiler settings".
* Add missing type parameters to overrides of ModRef.makeModVisitor
* Add missing type parameters to overrides of ModRef.makeRefVisitor
This also required slightly generalizing the generic type signature of
the ModRef.makeRefVisitor return type. Instead of returning
"RefVisitor<T, ExtendedHeapModel>", we now return "RefVisitor<T, ?
extends ExtendedHeapModel>". We need that extra flexibility because
at least one override of this method returns a RefVisitor whose second
generic type parameter is a subclass of ExtendedHeapModel.
Specifically, we're turning off Eclipse warnings about missing version
constraints on required bundles ("Require-Bundle"), exported
packages ("Export-Package"), and imported packages ("Import-Package").
We're not turning these off absolutely everywhere, though: only in
packages where one or more such warnings were actually being reported.
So if a given package was already providing all version constraints
for, say, package imports, then we've kept that warning on in that
package.
Honestly I don't entirely understand the practical implications of
these warnings. However, there were 355 of them across many WALA
subprojects. I take this as evidence that the WALA developers do not
consider these version constraints to be important. Therefore, we may
as well stop warning about something we have no intention of fixing.
That being said, if we *do* want to fix some or all of these, I
welcome any advice on what those fixes should look like. I am rather
ignorant about all things OSGi.
Removing an unused field sometimes means removing constructor code
that used to initialize that field. Removing that initialization code
sometimes leaves whole constructor arguments unused. Removing those
unused arguments can leave us with unused code to compute those
arguments in constructors' callers, and so on. This commit tries to
clean all of this up, working backward from the unused fields that an
earlier commit already removed. Hopefully I have avoided removing
upstream code that had other important side effects, but it wouldn't
hurt for a WALA expert to review this change carefully.
Manu requested that we use this approach instead of adding
`@SuppressWarnings("unused")` at each affected catch block. That
seems reasonable to me, given the large number of such warnings and
the lack of likely harm from ignoring such caught exceptions.
Fixing these Javadoc comments would require adding packages to various
other packages' build paths. In some of the cases suppressed,
changing build paths in that manner would create circular build
dependencies. In other cases, it would simply add a Javadoc-motivated
dependency that does not exist for the real code, which seems
undesirable. For a few cases, the reference seems to be to types in
code we don't even have here, such as code from "android" or
"org.mozilla" packages.
These arise, for example, when Javadoc documentation on a public class
includes a @link to a private field. I can see how this would be
problematic for closed-source Java code where private items are
invisible to outsiders. However, given that WALA is open source,
nothing is truly non-visible. If the WALA documentation authors
considered non-visible references useful when explaining things,
that's fine with me.
We don't turn this warning off in all projects. Rather, we turn it
off only in projects that were producing at least one such warning.
In other words, if a project was already completely "clean" with
respect to this warning, then we leave this warning enabled for that
project.
These changes turn off Eclipse warnings for Javadoc tags without
descriptions. In some subprojects, we turn these off entirely. In
others, leave on missing-descrption checks for "@return" tags only.
We don't turn this warning off in all projects. Rather, we turn it
off only in projects that were producing at least one such warning.
In other words, if a project was already completely "clean" with
respect to this warning, then we leave this warning enabled for that
project.
Turning off these warnings is a partial declaration of Javadoc
bankruptcy. In an ideal world, we would enable and fix all of these
warnings. However, there are 576 of them. Apparently the WALA team's
implicit coding style says that omitting descriptions is OK. If
there's no intent to systematically add descriptions, then we may as
well turn off these warnings so that we can see other warnings that we
may want to fix.
Note: some of these methods are decidedly nontrivial. Perhaps they
should not actually be removed? If any should be kept around, please
identify them to me. I'll revise this change to retain those methods
and simply annotate them as needed to suppress Eclipse's warning.
This fixes 33 out of 37 Eclipse "Potential resource leak: '...' may
not be closed" warnings. It also fixes 3 out of 37 Eclipse "Resource
'...' should be managed by try-with-resource" warnings, although that
was not the main focus of this effort.
The remaining 4 warnings about potential resource leaks all involve a
leaked JarFile instance that is passed to a JarFileModule constructor
call. JarFileModile never attempts to close its underlying JarFile;
this code is written as though JarFile cleanup were the caller's
responsibility. However, the JarFile often cannot be closed by the
code that creates the JarFileModule either, since the JarFile needs to
remain open while the JarFileModule is in use, and some of these
JarFileModules stay around beyond the lifetime of the code that
created them. Truly fixing this would essentially require making
JarFileModule implement Closeable, which in turn would probably
require that Module implement Closeable, which in turn would require
changes to lots of code that deals with Module instances to arrange
for them to be properly closed. That's more invasive than I'm
prepared to take on right now.
Instead, rely on Java's ability to infer type parameters in many
contexts. This removes 665 Eclipse warnings.
Note: a few of these changes are to files under "test" subdirectories.
Presumably those are files that serve as test inputs rather than being
part of WALA code proper. As far as I can tell, these changes do not
break any WALA tests. But if any of those tests were specifically
intended to exercise WALA on code with non-inferred generic type
parameters, then I really should be leaving those alone.
Previously, the various Eclipse projects' Java configurations used
mixtures of 1.6, 1.7, and 1.8. Many were internally inconsistent,
such as requiring 1.7 in "MANIFEST.MF" but 1.6 in the Eclipse JDT
build preferences. The Travis-CI configuration tests against both 1.7
and 1.8, but does not test against 1.6.
Across all projects, the most common version was 1.7. So I'm going to
assume that 1.7 is the intended build target. This commit makes 1.7
the selected version nearly everywhere.
"com.ibm.wala.core.testdata" is the one exception. This specific
project uses a few features only found in 1.8, such as lambda
expressions. Previously, "com.ibm.wala.core.testdata" used 1.7 in
some aspects of its configuration but 1.8 in others. Now it
consistently targets 1.8. I wish this one project didn't need to be
inconsistent with the rest of WALA, but at least now it's consistent
with itself.
(Personally, I'd be happy to target 1.8 only. But my impression
across all of these configuration files is that the WALA developers
still want to be compatible with 1.7. If that is no longer a
requirement, let me know and I will adjust these changes accordingly
to target 1.8 only.)
This change eliminates 11 "There is no 'jre.compilation.profile' build
entry and the project has Java compliance preferences set" warnings
and 13 "The JRE container on the classpath is not a perfect match to
the 'JavaSE-1.7' execution environment" warnings. However, it also
adds 450 "Redundant specification of type arguments <...>" warnings
and 17 "Resource '...' should be managed by try-with-resource"
warnings. So this seems like a net step backward in my wish to reduce
WALA warnings. However, those new warnings concern Java 1.7 language
features that we were not previously using to good effect in projects
that targeted 1.6. If we all agree that we can now target 1.7
instead, then we can use these helpful features as the newly-added
warnings suggest. So I call that a step in the right direction.
I think the "target/p2artifacts.xml" and "target/p2content.xml" files
are generated by Maven. They are well-formed XML but Eclipse's XML
validator legitimately warns that they lack grammar constraints.
Since we're not maintaining the tool that creates these files, we are
not in a position to do anything about that. Therefore, we may as
well exclude these from validation entirely. That way we can
more-clearly recognize warnings that we *can* do something about.
Ant "build.xml" files don't have a standard DTD or XML Schema; the
contents are simply too flexible for that. But we can at least
give each a stub DOCTYPE declaration. That's enough to satisfy
Eclipse's XML validator, which otherwise complains that these files
lack grammar constraints.
Eclipse validation warns about invalid HTML content in all
Maven-generated "target/site/dependency-convergence.html" files. The
warnings are legitimate: these HTML files are indeed invalid.
However, we don't maintain the tool that generates these files, so we
are not in a position to fix them. Better, therefore, to suppress
these warnings so that we can notice and fix other problems over which
we do have control.
In general, the WALA code base is not really ready for nullness
checking. It would be nice if we got there some day, but I'm not
planning to take that on now or any time soon. Until then, it's not
useful to warn about missing @NonNullByDefault declarations on WALA
packages.
See also older commit 7b6811b.
Eclipse Mars Service Release 2 finds 45 potential null pointer accesses
across WALA's various Eclipse projects. Eclipse ignores these by
default, but any individual user may have changed their personal Eclipse
configuration to treat them as warnings or errors. Thus, some people
will find that the code builds while others find that it fails. Better
to explicitly use a known-good configuration.
In the long run someone should inspect these cases one-by-one and fix
them where appropriate. But that is probably better managed as part of a
larger effort to tidy up nulls in WALA. I'm not planning to take that on
now or any time soon, though, so this is a better setup for now.
now, for me, code works using e44 with maven
dalvik tests refactored for mobile version with android dev tools
IDE tests Eclipse metadata fixed to make e44 work for me
new android entrypoint to fix failure in new droidbench tests
com.ibm.wala.cast.js.rhino.test/harness-src/com/ibm/wala/cast/js/test/TestPrototypeCallGraphShapeRhino.java
com.ibm.wala.cast.js.test/harness-src/com/ibm/wala/cast/js/test/TestPrototypeCallGraphShape.java
com.ibm.wala.cast.js.test.data/examples-src/pages/prototype.html
work (not yet finished) on fixes to property accesses for JavaScript:
com.ibm.wala.cast/source/java/com/ibm/wala/cast/ipa/callgraph/AstSSAPropagationCallGraphBuilder.java
com.ibm.wala.cast.java/src/com/ibm/wala/cast/java/ipa/callgraph/AstJavaSSAPropagationCallGraphBuilder.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/ipa/callgraph/JSSSAPropagationCallGraphBuilder.java
currently unused tests to remind me to fix bugs:
com.ibm.wala.cast.js.test/harness-src/com/ibm/wala/cast/js/test/TestSimpleCallGraphShape.java
com.ibm.wala.cast.js.test.data/examples-src/tests/loops.js
com.ibm.wala.cast.js.test.data/examples-src/tests/primitive_strings.js
fixes to exception handler code generation in JavaScript:
com.ibm.wala.cast.js.rhino/source/com/ibm/wala/cast/js/translator/RhinoToAstTranslator.java
com.ibm.wala.cast.js.test.data/examples-src/tests/try.js
com.ibm.wala.cast.js.test/harness-src/com/ibm/wala/cast/js/test/TestSimpleCallGraphShape.java
fixes to make the system build on both juno and luna
com.ibm.wala.cast.js.test.data/pom.xml
pom.xml
targets/e42/e42.target
targets/e44/e44.target
targets/pom.xml
com.ibm.wala.core.tests/META-INF/MANIFEST.MF
com.ibm.wala.dalvik.test/META-INF/MANIFEST.MF
com.ibm.wala.ide.jdt.test/META-INF/MANIFEST.MF
com.ibm.wala.ide.jdt/source/com/ibm/wala/cast/java/translator/jdt/FakeExceptionTypeBinding.java
com.ibm.wala.ide.jdt/source/com/ibm/wala/ide/util/JavaEclipseProjectPath.java
com.ibm.wala.ide.jsdt.tests/META-INF/MANIFEST.MF
com.ibm.wala.ide.jsdt.tests/src/com/ibm/wala/ide/jsdt/tests/AbstractJSProjectScopeTest.java
com.ibm.wala.ide/src/com/ibm/wala/ide/util/EclipseProjectPath.java
com.ibm.wala.ide/src/com/ibm/wala/ide/util/ProgressMonitorDelegate.java
beginnings of "pointer analysis" on top of field-based analysis
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/FlowGraph.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/vertices/PropVertex.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/vertices/RetVertex.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/vertices/VarVertex.java
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/callgraph/fieldbased/flowgraph/vertices/VertexFactory.java
com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/PointerAnalysis.java
com.ibm.wala.core/src/com/ibm/wala/ipa/callgraph/propagation/cfa/ExceptionReturnValueKey.java
fixes for crashes in correlartion tracking
com.ibm.wala.cast.js/source/com/ibm/wala/cast/js/ipa/callgraph/correlations/extraction/ClosureExtractor.java
fixes for Dalvik IR generation
com.ibm.wala.core/src/com/ibm/wala/cfg/BytecodeCFG.java
com.ibm.wala.core/src/com/ibm/wala/cfg/ShrikeCFG.java
com.ibm.wala.core/src/com/ibm/wala/ssa/SSACFG.java
com.ibm.wala.dalvik.test/source/com/ibm/wala/dalvik/drivers/APKCallGraphDriver.java
com.ibm.wala.dalvik.test/source/com/ibm/wala/dalvik/test/callGraph/JVMLDalvikComparison.java
com.ibm.wala.dalvik/src/com/ibm/wala/dalvik/classLoader/DexCFG.java
com.ibm.wala.dalvik/src/com/ibm/wala/dalvik/dex/instructions/UnaryOperation.java
com.ibm.wala.dalvik/src/com/ibm/wala/dalvik/ssa/AbstractIntRegisterMachine.java
com.ibm.wala.dalvik/src/com/ibm/wala/dalvik/ssa/DexSSABuilder.java
fixes to stack map generation when instrumenting for Java 7
com.ibm.wala.shrike/src/com/ibm/wala/shrike/cg/DynamicCallGraph.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/ConstantInstruction.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/analysis/Analyzer.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/analysis/ClassHierarchy.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/analysis/Verifier.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeBT/shrikeCT/ClassInstrumenter.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeCT/StackMapConstants.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeCT/StackMapTableReader.java
com.ibm.wala.shrike/src/com/ibm/wala/shrikeCT/StackMapTableWriter.java
analysis now understands and propagates MethodHandle objects
fixes to Shrike InvokeDynamic instruction
Former-commit-id: fb826f124423bcbca08f729cee1794fbda711d16
Ben Liblit
Julian Dolby
Achim D. Brucker
Manu Sridharan
Manu Sridharan
Julian Dolby
Martin Mohr
Manu Sridharan
Cosmin Radoi
Juergen Graf
Michael Heilmann