使用javax.tools.JavaCompiler在内存中完全编译代码

Question

我正在使用javax.tools包(JDK 1.7)中的JavaCompiler来动态编译一些东西,如下所示:

compiler.run(null, null, "-cp", paths, "path/to/my/file.java");

它可以工作,但我想在内存中完成所有操作(例如,传递带代码的字符串,而不是源文件,并获取字节代码而不是.class文件).我发现扩展InputStream和OutputStream参数是没有用的,因为它可能与控制台中的相同.你知道一种让run方法像这样工作的方法吗？或者您是否知道使用该getTask()方法确认的方法？(扩展FileManager看起来很简单但不是那么容易:)

Answer 1

我认为这可能是有帮助的,它基本上显示了如何从内存中编译Java源代码(字符串位于类中).

它使用PrinterWriter和StringWriter将源写入String/在内存中,然后使用JavaCompiler该类(从JDK 6开始)来编译和运行程序:

import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.JavaCompiler;
import javax.tools.JavaCompiler.CompilationTask;
import javax.tools.JavaFileObject;
import javax.tools.SimpleJavaFileObject;
import javax.tools.ToolProvider;
import java.io.File;
import java.io.IOException;
import java.io.PrintWriter;
import java.io.StringWriter;
import java.lang.reflect.InvocationTargetException;
import java.net.URI;
import java.net.URL;
import java.net.URLClassLoader;
import java.util.Arrays;


public class CompileSourceInMemory {
  public static void main(String args[]) throws IOException {
    JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
    DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<JavaFileObject>();

    StringWriter writer = new StringWriter();
    PrintWriter out = new PrintWriter(writer);
    out.println("public class HelloWorld {");
    out.println("  public static void main(String args[]) {");
    out.println("    System.out.println(\"This is in another java file\");");    
    out.println("  }");
    out.println("}");
    out.close();
    JavaFileObject file = new JavaSourceFromString("HelloWorld", writer.toString());

    Iterable<? extends JavaFileObject> compilationUnits = Arrays.asList(file);
    CompilationTask task = compiler.getTask(null, null, diagnostics, null, null, compilationUnits);

    boolean success = task.call();
    for (Diagnostic diagnostic : diagnostics.getDiagnostics()) {
      System.out.println(diagnostic.getCode());
      System.out.println(diagnostic.getKind());
      System.out.println(diagnostic.getPosition());
      System.out.println(diagnostic.getStartPosition());
      System.out.println(diagnostic.getEndPosition());
      System.out.println(diagnostic.getSource());
      System.out.println(diagnostic.getMessage(null));

    }
    System.out.println("Success: " + success);

    if (success) {
      try {

          URLClassLoader classLoader = URLClassLoader.newInstance(new URL[] { new File("").toURI().toURL() });
          Class.forName("HelloWorld", true, classLoader).getDeclaredMethod("main", new Class[] { String[].class }).invoke(null, new Object[] { null });

      } catch (ClassNotFoundException e) {
        System.err.println("Class not found: " + e);
      } catch (NoSuchMethodException e) {
        System.err.println("No such method: " + e);
      } catch (IllegalAccessException e) {
        System.err.println("Illegal access: " + e);
      } catch (InvocationTargetException e) {
        System.err.println("Invocation target: " + e);
      }
    }
  }
}

class JavaSourceFromString extends SimpleJavaFileObject {
  final String code;

  JavaSourceFromString(String name, String code) {
    super(URI.create("string:///" + name.replace('.','/') + Kind.SOURCE.extension),Kind.SOURCE);
    this.code = code;
  }

  @Override
  public CharSequence getCharContent(boolean ignoreEncodingErrors) {
    return code;
  }
}

如果您查看参考链接,您还可以找到更多其他示例

参考:

• 从内存编译

Answer 2

我在Mac OS Java 7中运行了上述代码.它们都不起作用.所以我写了一个 https://github.com/trung/InMemoryJavaCompiler

StringBuffer sourceCode = new StringBuffer();
sourceCode.append("package org.mdkt;\n");
sourceCode.append("public class HelloClass {\n");
sourceCode.append("   public String hello() { return \"hello\"; }");
sourceCode.append("}");

Class<?> helloClass = InMemoryJavaCompiler.compile("org.mdkt.HelloClass", sourceCode.toString());

Answer 3

这是一个完全在内存中编译的类.

我从Rekha Kumari(2011年6月)的http://javapracs.blogspot.de/2011/06/dynamic-in-memory-compilation-using.html获取了(几乎)全部内容.虽然这个版本缩短了100多行,并且具有更多功能(但没有文档:P).

它可以一次编译多个类,这是编译彼此依赖的类的唯一方法.如果你想知道"CompilerFeedback"这个类:我正在开发一个小型Java IDE,用于我需要的编码游戏.我在这里包含它是因为我假设您想要对此编译器执行某些操作,并且预消化可能有助于此.(我意识到CompilerFeedback类中的一些代码是完整的废话.它是从多年的尝试中回收的.

还有一个不需要编译的实用方法,它从类的源代码(包括包名,如果提供的话)中派生出完整的类名.对于调用编译器非常有用,它确实需要这些信息.

演示课程:

import java.util.ArrayList;
import java.util.List;

public class Demo {

    public static void main(final String[] args) {

        final InMemoryCompiler.IMCSourceCode cls1source;
        final InMemoryCompiler.IMCSourceCode cls2source;

        final StringBuilder sb = new StringBuilder();
        sb.append("package toast;\n");
        sb.append("public class DynaClass {\n");
        sb.append("    public static void main(final String[] args) {");
        sb.append("        System.out.println(\"Based massively on the work of Rekha Kumari, http://javapracs.blogspot.de/2011/06/dynamic-in-memory-compilation-using.html\");\n");
        sb.append("        System.out.println(\"This is the main method speaking.\");\n");
        sb.append("        System.out.println(\"Args: \" + java.util.Arrays.toString(args));\n");
        sb.append("        final Test test = new Test();\n");
        sb.append("    }\n");
        sb.append("    public String toString() {\n");
        sb.append("        return \"Hello, I am \" + ");
        sb.append("this.getClass().getSimpleName();\n");
        sb.append("    }\n");
        sb.append("}\n");
        cls1source = new InMemoryCompiler.IMCSourceCode("toast.DynaClass", sb.toString());

        sb.setLength(0);
        sb.append("package toast;\n");
        sb.append("public class Test {\n");
        sb.append("    public Test() {\n");
        sb.append("        System.out.println(\"class Test constructor reporting in.\");\n");
        sb.append("        System.out.println(new DynaClass());\n");
        sb.append("    }\n");
        sb.append("}\n");
        cls2source = new InMemoryCompiler.IMCSourceCode("toast.Test", sb.toString());

        final List<InMemoryCompiler.IMCSourceCode> classSources = new ArrayList<>();
        classSources.add(cls1source);
        classSources.add(cls2source);

        final InMemoryCompiler uCompiler = new InMemoryCompiler(classSources);
        final CompilerFeedback compilerFeedback = uCompiler.compile();
        System.out.println("\n\nCOMPILER FEEDBACK: " + compilerFeedback);

        if (compilerFeedback != null && compilerFeedback.success) {

            try {
                System.out.println("\nTOSTRING DEMO:");
                uCompiler.runToString(cls1source.fullClassName);
            } catch (Exception e) {
                e.printStackTrace();
            }

            try {
                System.out.println("\nMAIN DEMO:");
                uCompiler.runMain(cls1source.fullClassName, new String[] { "test1", "test2" });
            } catch (Exception e) {
                e.printStackTrace();
            }
        }
        System.exit(0);
    }
}

编译器类:

import javax.tools.*;
import java.io.ByteArrayOutputStream;
import java.io.IOException;
import java.io.OutputStream;
import java.lang.reflect.InvocationTargetException;
import java.lang.reflect.Method;
import java.net.URI;
import java.security.SecureClassLoader;
import java.util.ArrayList;
import java.util.HashMap;
import java.util.List;
import java.util.Map;

/**
 * MASSIVELY based on http://javapracs.blogspot.de/2011/06/dynamic-in-memory-compilation-using.html by Rekha Kumari
 * (June 2011)
 */
final public class InMemoryCompiler {

    final public static class IMCSourceCode {

        final public String fullClassName;
        final public String sourceCode;

        /**
         * @param fullClassName Full name of the class that will be compiled. If the class should be in some package,
         *                      fullName should contain it too, for example: "testpackage.DynaClass"
         * @param sourceCode    the source code
         */
        public IMCSourceCode(final String fullClassName, final String sourceCode) {

            this.fullClassName = fullClassName;
            this.sourceCode = sourceCode;
        }
    }

    final public boolean valid;

    final private List<IMCSourceCode> classSourceCodes;
    final private JavaFileManager fileManager;

    public InMemoryCompiler(final List<IMCSourceCode> classSourceCodes) {

        this.classSourceCodes = classSourceCodes;

        final JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();
        if (compiler == null) {
            fileManager = null;
            valid = false;
            System.err.println("ToolProvider.getSystemJavaCompiler() returned null! This program needs to be run on a system with an installed JDK.");
            return;
        }
        valid = true;

        fileManager = new ForwardingJavaFileManager<JavaFileManager>(compiler.getStandardFileManager(null, null, null)) {

            final private Map<String, ByteArrayOutputStream> byteStreams = new HashMap<>();

            @Override
            public ClassLoader getClassLoader(final Location location) {

                return new SecureClassLoader() {

                    @Override
                    protected Class<?> findClass(final String className) throws ClassNotFoundException {

                        final ByteArrayOutputStream bos = byteStreams.get(className);
                        if (bos == null) {
                            return null;
                        }
                        final byte[] b = bos.toByteArray();
                        return super.defineClass(className, b, 0, b.length);
                    }
                };
            }

            @Override
            public JavaFileObject getJavaFileForOutput(final Location location, final String className, final JavaFileObject.Kind kind, final FileObject sibling) throws IOException {

                return new SimpleJavaFileObject(URI.create("string:///" + className.replace('.', '/') + kind.extension), kind) {

                    @Override
                    public OutputStream openOutputStream() throws IOException {

                        ByteArrayOutputStream bos = byteStreams.get(className);
                        if (bos == null) {
                            bos = new ByteArrayOutputStream();
                            byteStreams.put(className, bos);
                        }
                        return bos;
                    }
                };
            }
        };
    }

    public CompilerFeedback compile() {

        if (!valid) {
            return null;
        }
        final List<JavaFileObject> files = new ArrayList<>();
        for (IMCSourceCode classSourceCode : classSourceCodes) {
            URI uri = null;
            try {
                uri = URI.create("string:///" + classSourceCode.fullClassName.replace('.', '/') + JavaFileObject.Kind.SOURCE.extension);
            } catch (Exception e) {
                //                e.printStackTrace();
            }
            if (uri != null) {
                final SimpleJavaFileObject sjfo = new SimpleJavaFileObject(uri, JavaFileObject.Kind.SOURCE) {

                    @Override
                    public CharSequence getCharContent(final boolean ignoreEncodingErrors) {

                        return classSourceCode.sourceCode;
                    }
                };
                files.add(sjfo);
            }
        }

        final DiagnosticCollector<JavaFileObject> diagnostics = new DiagnosticCollector<>();

        final JavaCompiler compiler = ToolProvider.getSystemJavaCompiler();

        if (files.size() > 0) {
            final JavaCompiler.CompilationTask task = compiler.getTask(null, fileManager, diagnostics, null, null, files);
            return new CompilerFeedback(task.call(), diagnostics);
        } else {
            return null;
        }
    }

    public void runToString(final String className) throws InstantiationException, IllegalAccessException, ClassNotFoundException {

        if (!valid) {
            return;
        }
        final Class<?> theClass = getCompiledClass(className);
        final Object instance = theClass.newInstance();
        System.out.println(instance);
    }

    public void runMain(final String className, final String[] args) throws IllegalAccessException, ClassNotFoundException, NoSuchMethodException, InvocationTargetException {

        if (!valid) {
            return;
        }
        final Class<?> theClass = getCompiledClass(className);
        final Method mainMethod = theClass.getDeclaredMethod("main", String[].class);
        mainMethod.invoke(null, new Object[] { args });
    }

    public Class<?> getCompiledClass(final String className) throws ClassNotFoundException {

        if (!valid) {
            throw new IllegalStateException("InMemoryCompiler instance not usable because ToolProvider.getSystemJavaCompiler() returned null: No JDK installed.");
        }
        final ClassLoader classLoader = fileManager.getClassLoader(null);
        final Class<?> ret = classLoader.loadClass(className);
        if (ret == null) {
            throw new ClassNotFoundException("Class returned by ClassLoader was null!");
        }
        return ret;
    }
}

COMPILERFEEDBACK CLASS:

import javax.tools.Diagnostic;
import javax.tools.DiagnosticCollector;
import javax.tools.JavaFileObject;
import javax.tools.SimpleJavaFileObject;
import java.io.IOException;
import java.util.ArrayList;
import java.util.List;
import java.util.Locale;

final public class CompilerFeedback {

    final public boolean success;
    final public List<CompilerMessage> messages = new ArrayList<>();

    public CompilerFeedback(final Boolean success, final DiagnosticCollector<JavaFileObject> diagnostics) {

        this.success = success != null && success;
        for (Diagnostic<? extends JavaFileObject> diagnostic : diagnostics.getDiagnostics()) {
            messages.add(new CompilerMessage(diagnostic));
        }
    }

    public String toString() {

        final StringBuilder sb = new StringBuilder();

        sb.append("SUCCESS: ").append(success).append('\n');
        final int iTop = messages.size();
        for (int i = 0; i < iTop; i++) {
            sb.append("\n[MESSAGE ").append(i + 1).append(" OF ").append(iTop).append("]\n\n");
            // sb.append(messages.get(i).toString()).append("\n");
            // sb.append(messages.get(i).toStringForList()).append("\n");
            sb.append(messages.get(i).toStringForDebugging()).append("\n");
        }
        return sb.toString();
    }

    final public static class CompilerMessage {

        final public Diagnostic<? extends JavaFileObject> compilerInfo;

        final public String typeOfProblem;
        final public String typeOfProblem_forDebugging;

        final public String multiLineMessage;

        final public int lineNumber;
        final public int columnNumber;

        final public int textHighlightPos_lineStart;
        final public int textHighlightPos_problemStart;
        final public int textHighlightPos_problemEnd;

        final public String sourceCode;
        final public String codeOfConcern;
        final public String codeOfConcernLong;

        CompilerMessage(final Diagnostic<? extends JavaFileObject> diagnostic) {

            final JavaFileObject sourceFileObject = diagnostic.getSource();
            String sourceCodePreliminary = null;
            if (sourceFileObject instanceof SimpleJavaFileObject) {
                final SimpleJavaFileObject simpleSourceFileObject = (SimpleJavaFileObject) sourceFileObject;

                try {
                    final CharSequence charSequence = simpleSourceFileObject.getCharContent(false);
                    sourceCodePreliminary = charSequence.toString();
                } catch (IOException e) {
                    e.printStackTrace();
                }
            }
            if (sourceCodePreliminary == null) {
                sourceCode = "[SOURCE CODE UNAVAILABLE]";
            } else {
                sourceCode = sourceCodePreliminary;
            }

            compilerInfo = diagnostic;

            typeOfProblem = diagnostic.getKind().name();
            typeOfProblem_forDebugging = "toString() = " + diagnostic.getKind().toString() + "; name() = " + typeOfProblem;

            lineNumber = (int) compilerInfo.getLineNumber();
            columnNumber = (int) compilerInfo.getColumnNumber();

            final int sourceLen = sourceCode.length();
            textHighlightPos_lineStart = (int) Math.min(Math.max(0, diagnostic.getStartPosition()), sourceLen);
            textHighlightPos_problemStart = (int) Math.min(Math.max(0, diagnostic.getPosition()), sourceLen);
            textHighlightPos_problemEnd = (int) Math.min(Math.max(0, diagnostic.getEndPosition()), sourceLen);

            final StringBuilder reformattedMessage = new StringBuilder();
            final String message = diagnostic.getMessage(Locale.US);
            final int messageCutOffPosition = message.indexOf("location:");
            final String[] messageParts;
            if (messageCutOffPosition >= 0) {
                messageParts = message.substring(0, messageCutOffPosition).split("\n");
            } else {
                messageParts = message.split("\n");
            }
            for (String s : messageParts) {
                String s2 = s.trim();
                if (s2.length() > 0) {
                    boolean lengthChanged;
                    do {
                        final int lBeforeReplace = s2.length();
                        s2 = s2.replace("  ", " ");
                        lengthChanged = (s2.length() != lBeforeReplace);
                    } while (lengthChanged);
                    reformattedMessage.append(s2).append("\n");
                }
            }

            codeOfConcern = sourceCode.substring(textHighlightPos_problemStart, textHighlightPos_problemEnd);
            codeOfConcernLong = sourceCode.substring(textHighlightPos_lineStart, textHighlightPos_problemEnd);
            if (!codeOfConcern.isEmpty()) {
                reformattedMessage.append("Code of concern: \"").append(codeOfConcern).append('\"');
            }
            multiLineMessage = reformattedMessage.toString();
        }

        public String toStringForList() {

            if (compilerInfo == null) {
                return "No compiler!";
            } else {
                return compilerInfo.getCode();
            }
        }

        public String toStringForDebugging() {

            final StringBuilder ret = new StringBuilder();

            ret.append("Type of problem: ").append(typeOfProblem_forDebugging).append("\n\n");
            ret.append("Message:\n").append(multiLineMessage).append("\n\n");

            ret.append(compilerInfo.getCode()).append("\n\n");

            ret.append("line number: ").append(lineNumber).append("\n");
            ret.append("column number: ").append(columnNumber).append("\n");

            ret.append("textHighlightPos_lineStart: ").append(textHighlightPos_lineStart).append("\n");
            ret.append("textHighlightPos_problemStart: ").append(textHighlightPos_problemStart).append("\n");
            ret.append("textHighlightPos_problemEnd: ").append(textHighlightPos_problemEnd).append("\n");

            return ret.toString();
        }

        @Override
        public String toString() {

            //            return compilerInfo.getMessage(Locale.US);
            return typeOfProblem + ": " + multiLineMessage + "\n";
        }
    }
}

实用方法(三个类不需要进一步提升.):

final public static String PREFIX_CLASSNAME = "class ";
final public static String PREFIX_PACKAGENAME = "package ";
final public static String CHARSET_JAVAKEYWORDENDERS = " \n[](){}<>;,\"\\/*+-=%!&?@:";

/**
 * @return e.g. "com.dreamspacepresident.TestClass" if the source's first root level "class" (I'm talking about {}
 * hierarchy.) is named "TestClass", and if the "package" name is "com.dreamspacepresident". Null is returned if
 * sourceCode is null or does not provide a class name. (Mind that the parsing is done in a quite crappy way.)
 */
public static String deriveFullClassNameFromSource(final String sourceCode) {

    if (sourceCode == null) {
        return null;
    }
    final int firstBr = sourceCode.indexOf('{');
    if (firstBr >= 0) {
        // DETERMINE CLASS NAME
        final int firstClass = sourceCode.indexOf(PREFIX_CLASSNAME);
        if (firstClass < firstBr) {
            String className = sourceCode.substring(firstClass + PREFIX_CLASSNAME.length(), firstBr).trim();
            final int classNameEnd = indexOfAnyOfThese(className, CHARSET_JAVAKEYWORDENDERS);
            if (classNameEnd >= 0) {
                className = className.substring(0, classNameEnd);
            }
            if (!className.isEmpty()) {
                // DETERMINE PACKAGE NAME
                String packageName = null;
                final int firstPackage = sourceCode.indexOf(PREFIX_PACKAGENAME);
                if (firstPackage >= 0 && firstPackage < firstBr && firstPackage < firstClass) {
                    packageName = sourceCode.substring(firstPackage + PREFIX_PACKAGENAME.length(), firstBr).trim();
                    final int packageNameEnd = indexOfAnyOfThese(packageName, CHARSET_JAVAKEYWORDENDERS);
                    if (packageNameEnd >= 0) {
                        packageName = packageName.substring(0, packageNameEnd);
                    }
                }
                return (packageName != null && !packageName.isEmpty() ? packageName + "." : "") + className;
            }
        }
    }
    return null;
}


/**
 * Looks for the first occurrence of ANY of the given characters, which is easier than using a bunch of
 * String.indexOf() calls.
 *
 * @return -1 if not found, otherwise the String index of the first hit
 */
public static int indexOfAnyOfThese(final String text, final String characters) {

    if (text != null && !text.isEmpty() && characters != null && !characters.isEmpty()) {
        final int lenT = text.length();
        final int lenC = characters.length();
        for (int i = 0; i < lenT; i++) {
            final char c = text.charAt(i);
            for (int ii = 0; ii < lenC; ii++) {
                if (c == characters.charAt(ii)) {
                    return i;
                }
            }
        }
    }
    return -1;
}

Answer 4

几年前我写了一个库来做这件事.它需要一个String,它可以包含嵌套类,编译它们并可选地将它们加载到当前的类加载器中(因此您不需要额外的类加载器)如果JVM在调试模式下运行,它会将生成的代码写入文件这样您就可以逐步完成生成的代码.

http://vanillajava.blogspot.co.uk/2010_11_01_archive.html

用erolagnab你的例子来解释这个例子

StringBuilder sourceCode = new StringBuilder();
sourceCode.append("package org.mdkt;\n")
          .append("public class HelloClass {\n")
          .append("   public String hello() { return \"hello\"; }")
          .append("}");

Class<?> helloClass = CACHED_COMPILER.compile("org.mdkt.HelloClass",
                                              sourceCode.toString());

更新,源代码可在此处获得https://github.com/OpenHFT/Java-Runtime-Compiler

您可以通过maven http://search.maven.org/#browse%7C842970587获取最新版本

一个较长的例子.

// this writes the file to disk only when debugging is enabled.
CachedCompiler cc = CompilerUtils.DEBUGGING ?
        new CachedCompiler(new File(parent, "src/test/java"), new File(parent, "target/compiled")) :
        CompilerUtils.CACHED_COMPILER;

String text = "generated test " + new Date();
Class fooBarTeeClass = cc.loadFromJava("eg.FooBarTee", "package eg;\n" +
    '\n' +
    "import eg.components.BarImpl;\n" +
    "import eg.components.TeeImpl;\n" +
    "import eg.components.Foo;\n" +
    '\n' +
    "public class FooBarTee{\n" +
    "    public final String name;\n" +
    "    public final TeeImpl tee;\n" +
    "    public final BarImpl bar;\n" +
    "    public final BarImpl copy;\n" +
    "    public final Foo foo;\n" +
    '\n' +
    "    public FooBarTee(String name) {\n" +
    "        // when viewing this file, ensure it is synchronised with the copy on disk.\n" +
    "        System.out.println(\"" + text + "\");\n" +
    "        this.name = name;\n" +
    '\n' +
    "        tee = new TeeImpl(\"test\");\n" +
    '\n' +
    "        bar = new BarImpl(tee, 55);\n" +
    '\n' +
    "        copy = new BarImpl(tee, 555);\n" +
    '\n' +
    "        // you should see the current date here after synchronisation.\n" +
    "        foo = new Foo(bar, copy, \"" + text + "\", 5);\n" +
    "    }\n" +
    '\n' +
    "    public void start() {\n" +
    "    }\n" +
    '\n' +
    "    public void stop() {\n" +
    "    }\n" +
    '\n' +
    "    public void close() {\n" +
    "        stop();\n" +
    '\n' +
    "    }\n" +
    "}\n");

// add a debug break point here and step into this method.
FooBarTee fooBarTee = new FooBarTee("test foo bar tee");
Foo foo = fooBarTee.foo;
assertNotNull(foo);
assertEquals(text, foo.s);