Commit Graph

5997 Commits

Author SHA1 Message Date
Ben Liblit c034dce3c3 Allow parallel testing within each subproject
Without this setting, Gradle would run multiple "test" tasks from
multiple subprojects concurrently, but each "test" task would only run
a single test at a time.  Some of our subprojects' test tasks take a
long time to complete, which could easily leave us sequentially
testing on just one or two CPU cores while other cores sit idle.

With this change, Gradle will use as many task slots as are
available (e.g., when run with "--parallel") to run multiple
simultaneous JUnit test classes within any given subproject.  This
seems to be fine for us:  I am unaware of any shared global state that
would cause conflicts between two test classes within any given
subproject.
2018-04-17 15:02:36 -05:00
Ben Liblit 3f5460231e Don't try to exhaustively anticipate all generated header names
When JNI headers for a given class, each nested class will end up with
its own additional header.  But I don't want to try to parse nested
class details out of the Java source files just so we can determine
exactly which headers will be created.  Instead, simply treat the
entire header destination directory as the output of this task.
2018-04-17 15:02:36 -05:00
Ben Liblit 66a4d20e3a Improve build caching by reducing path sensitivity where appropriate 2018-04-17 15:02:36 -05:00
Ben Liblit 6869a4d5f4 Enable caching of task outputs for custom build tasks 2018-04-17 15:02:36 -05:00
Ben Liblit e42e36c4de Rename a build task to more clearly reflect what it builds
This task has an input named "hello_hash.ml", and an output named
"hello_hash.jar".  So calling this task "generateHelloHash" is too
vague.  Now we call it "generateHelloHashJar" instead.
2018-04-17 15:02:36 -05:00
Ben Liblit 4f94b4c341 Extend Gradle's "clean" task to remove some additional build targets 2018-04-17 15:02:36 -05:00
Ben Liblit b4fca9c38d Merge info from existing, hand-authored manifests when building jars 2018-04-17 15:02:36 -05:00
Ben Liblit 272e3adaf8 Tell Eclipse to ignore some questionable manifest contents
These manifest files are here for use by the Maven build, but Eclipse
is now using Gradle (via Buildship).  So the manifests as seen by
Eclipse do not entirely make sense.  I'm hesitant to change the
manifests directly, since presumably they were correct for Maven and
still are.

Perhaps some day we'll be using Gradle to generate manifests.  Until
that day comes, we're better off leaving the manifests as they are and
just suppressing Eclipse's warnings instead.
2018-04-17 15:02:36 -05:00
Ben Liblit 9d7df5de7e After Builldship import, put "dx.jar" where the manifest expects it 2018-04-17 15:02:36 -05:00
Ben Liblit 7a5f140127 Restore old manifests in hope of fixing Maven regressions 2018-04-17 15:02:36 -05:00
Ben Liblit 4a707d954a Bail out if any Travis CI testing commands fail
Previously we could fail some "mvn" stage but keep running anyway,
thereby fooling us into thinking that everything was OK.
2018-04-17 15:02:36 -05:00
Ben Liblit 08dc11ad7d Incremental improvements to Windows JDK library configuration
This still doesn't actually work, but it's closer than it was before.
There's still some problem with improper mixing of 32-bit ("x86") and
64-bit ("x64") libraries.
2018-04-17 15:02:36 -05:00
Ben Liblit e8b86fdf82 Clean up questionable memory management during string construction
Avoid allocating memory using strdup() and then releasing it using
operator delete.  strdup()-allocated memory should be released using
free(), not delete.  But in these specific cases, there really was
never any good reason to duplicate the C-style strings in the first
place.  Instead, we can safely pass those NUL-terminated char pointers
directly in calls to JNI's NewStringUTF().  NewStringUTF() does not
take ownership of its argument, but rather clones that string data
internally before returning.  So using strdup() and delete was just
unnecessary memory churn.

In cases where we need to format, concatenate, or construct new
strings, don't use sprintf() into variable-sized, stack-allocated
arrays.  Variable-sized arrays are not portable, and in particular are
rejected by Microsoft's Visual Studio 2017 C++ compiler.  Instead, do
our string formatting and manipulations using std::ostringstream
and/or std::string.  We just need to be a bit careful about the
lifetimes and ownership responsibilities of allocated data.  In
brief, (1) ostringstream::str() returns a temporary string instance
that expires at the end of the enclosing statement, independent of the
lifetime of the ostringstream instance; while (2) string::c_str()
returns an pointer to internal data that remains valid as long as the
string on which it was called is valid and unmodified.
2018-04-17 15:02:36 -05:00
Ben Liblit 4b04f8d812 Somebody decided to use three-space indentation in this file only
OK, whatever makes you happy.
2018-04-17 15:02:36 -05:00
Ben Liblit e9fa6f31b3 Fill in likely (but untested) JVM library path on Windows 2018-04-17 15:02:36 -05:00
Ben Liblit 0a21f20348 Factor out common code for installing Android SDK components 2018-04-17 15:02:36 -05:00
Ben Liblit ad111547de Skip DroidBench tests on Windows, for now
The default location of DroidBench in "/tmp/DroidBench" does not work
well on Windows. So let's disable these tests until someone has time to
make that path more portable.
2018-04-17 15:02:36 -05:00
Ben Liblit f568db5de6 Fix Android SDK installation under Windows 2018-04-17 15:02:36 -05:00
Ben Liblit 7f8de6d611 Switch to a different DroidBench download URL
This URL skips over a redirect that the previous URL went through.
This URL also avoids an annoying "Invalid cookie header" warning that
the previous URL produced.
2018-04-17 15:02:36 -05:00
Ben Liblit e4f9fedf38 Use download task instead of "git clone" to get DroidBench
This gives us better progress reporting during the download, better
caching, better file-level dependency tracking, etc.
2018-04-17 15:02:36 -05:00
Ben Liblit 813c79caa0 Factor out recurring pattern in download tasks 2018-04-17 15:02:36 -05:00
Ben Liblit 4749fe47c0 Experimental combined task for download and checksum verification
Not working yet:
<https://github.com/michel-kraemer/gradle-download-task/issues/108>.
2018-04-17 15:02:36 -05:00
Ben Liblit ba455f4737 Update gradle-download-plugin and use some of its new features
We now download and verify checksums as a single task, rather than as
two separate tasks.  This simplifies other task dependencies, since we
no longer have a checksum-verified "stamp" file separate from the
download itself.  Unfortunately the combined task now has a
significant amount of repeated boilerplate.  I'm hoping to refactor
that all out into a custom task class, but haven't yet figured out the
details:
<https://github.com/michel-kraemer/gradle-download-task/issues/108>.

We now also use ETags to be smarter about when a fresh download is or
is not actually needed.  I think there are still opportunities for
improved caching here, but this is a step in the right direction.
2018-04-17 15:02:36 -05:00
Ben Liblit 17f42c3725 If Travis CI gives us multiple CPUs, we may as well use them
I believe Travis CI jobs get two CPUs by default.

Doing parallel builds regularly is also a good way to help us discover
any build race conditions we may have.  There's no guarantee that any
such races will be revealed, but even exposing them
nondeterministically is better than having no possibility of exposing
them at all.
2018-04-17 15:02:36 -05:00
Ben Liblit 96055cca75 Fix a shellcheck warning when setting a variable to the empty string 2018-04-17 15:02:36 -05:00
Ben Liblit 00a645100e Update to Gradle 4.6
No significant new features of interest for us in this release, but I
like to keep current.
2018-04-17 15:02:36 -05:00
Ben Liblit 596c9091bb Add a new README with Gradle-specific instructions and tips 2018-04-17 15:02:36 -05:00
Ben Liblit 29fad5b1e0 Remove two Maven repositories that we no longer use for anything
We used to use these to find various Eclipse packages, but that was
always a dodgy affair since we never quite knew whether we had
matching versions of everything.  Now that we are using the
"com.diffplug.gradle.p2.asmaven" plug-in, though, we have much better
control over getting exactly the Eclipse material we need.  These two
Maven repositories no longer provide anything we use, and therefore
can be removed.
2018-04-17 15:02:36 -05:00
Ben Liblit 43f7704d5d We only expect ".gradle" directories in two specific places 2018-04-17 15:02:36 -05:00
Ben Liblit 84f1977ab4 Generalize post-import Eclipse run configuration for more fixups
Previously this launcher's job was to run "processTestResources" and
any other Gradle tasks needed to create files that Eclipse was
expecting to be available.  But we also want to use it to revert the
bad changes that Buildship applies to ".launch" configuration files.

This is a temporary hack to work around
<https://github.com/eclipse/buildship/issues/653>.
2018-04-17 15:02:36 -05:00
Ben Liblit 5b1aed99a7 Minor indentation fix 2018-04-17 15:02:36 -05:00
Ben Liblit faf3b86e1b Extra help for finding "cast" native library on macOS
Unlike Linux, macOS has no "RPATH" facility for embedding additional
search paths within shared libraries.  Instead, we need to set
"DYLD_LIBRARY_PATH" appropriately in the environment.  This
environment variable is the macOS analogue of "LD_LIBRARY_PATH" on
Linux.

Note that adding the required path to the "java.library.path" system
property will *not* work.  This property only affects where the JVM
looks for shared objects that it is loading directly.  This property
does not influence the search for other, transitively-required shared
objects.

Fixes #3.
2018-04-17 15:02:36 -05:00
Ben Liblit cac0e87082 Fix remaining Javadoc warnings
This fixes the last of our Javadoc warnings without creating a
circular dependency between ":com.ibm.wala.cast:javadoc" and
":com.ibm.wala.cast.js:javadoc".  Fixes #4, wherein more details about
this tricky dependency challenge can be found.
2018-04-17 15:02:36 -05:00
Ben Liblit 709c38c049 Use ".singleFile" instead of "[0]" when we expect just one file
This documents our intent more clearly, and actively checks for
mistakes rather than silently ignoring unexpected extra files.
2018-04-17 15:02:36 -05:00
Ben Liblit e4f8d00cbe Start exploring publication features by adding source jar archives 2018-04-17 15:02:36 -05:00
Ben Liblit 2bf92e1214 Replicate test include/exclude patterns already used in Maven builds 2018-04-17 15:02:36 -05:00
Ben Liblit f287ee5c41 Add "com.ibm.wala.core.testdata" to classpath for testing
This lets the "DynamicCallGraphTest" tests pass.  The tests in that
class expect to find some element of the classpath that includes the
substring "com.ibm.wala.core.testdata".  They then treat that as a
collection of bytecode files to instrument.
2018-04-17 15:02:36 -05:00
Ben Liblit b8f1bb3aaf Skip a test intentionally; Maven already does the same
Big thanks to Julian for showing me where this exclusion logic lives
in the Maven configuration.  There's a "**/*AndroidLibs*.java"
exclusion pattern in the top-level "pom.xml".
2018-04-17 15:02:36 -05:00
Ben Liblit 2763589d1f Fix jar exclusion patterns to match what Ant uses
One previously-failing test class now passes.  Progress!
2018-04-17 15:02:36 -05:00
Ben Liblit d301a89f4f Add some Eclipse dependencies needed only when running tests
This allows two test classes to pass that previously were failing.
Yay!
2018-04-17 15:02:36 -05:00
Ben Liblit 06d464216b Add a resource directory containing a zip archive used for testing
The excluded test in this subproject still does not pass, though now
it's failing for a different reason than before.  Progress, I suppose!
2018-04-17 15:02:36 -05:00
Ben Liblit 7227deb8a9 Disable Gradle linter check for unused dependencies in Travis CI
It's telling me to remove "eclipse-deps:org.eclipse.core.runtime:+"
and "org.osgi:org.osgi.core:4.2.0" as unused dependencies in the
"com.ibm.wala.cast.java.ecj" subproject.  However, these two
dependencies (jar files) are actually needed; compilation fails
without them.
2018-04-17 15:02:36 -05:00
Ben Liblit 7e0ac84a03 Also get Eclipse JSDT from a P2 repository, but not the same one 2018-04-17 15:02:36 -05:00
Ben Liblit 5e0e251766 Find Eclipse jars using local Maven mirror of Eclipse P2 repository
This should give us a set of mutually-consistent jars rather than
picking up random, outdated pieces from Maven Central or wherever else
I could find them.  We now also have a single, central place where we
set the Eclipse version that we're building against.  Much, *much*
cleaner.
2018-04-17 15:02:36 -05:00
Ben Liblit 196dd2d971 Assume that Eclipse needs all standard test-resource processing 2018-04-17 15:02:36 -05:00
Ben Liblit d230fe836c Trivial whitespace tweak 2018-04-17 15:02:36 -05:00
Ben Liblit eb2d2dd850 Exclude one macOS test that I don't know how to fix right now 2018-04-17 15:02:36 -05:00
Ben Liblit e9ea4d7f1d Download appropriate Android SDK for current OS 2018-04-17 15:02:36 -05:00
Ben Liblit c02776fc4d Try to continue building/testing even after the first failure
If other, non-dependent tasks would also fail, I'd rather discover all
of them at once.
2018-04-17 15:02:36 -05:00
Ben Liblit fb33cbe03b "rm" on macOS apparently doesn't understand long ("--foo") flags 2018-04-17 15:02:36 -05:00