Hadoop的2.5.2 mvn构build失败Windows 7的x64 – 编译MS-native-dll的hadoop常见?

我在这里跟着Hadoop2OnWindows的Wiki,我已经多次阅读了源代码树中的BUILDING.txt 。 我想我有所有的依赖关系了,但我得到一个奇怪的Failed to execute goal (compile-ms-native-dll)错误(完整日志下面)。

我已经安装了Windows SDK 7.1

以下是我的path:

  • Cygwin/bin
  • MSBuild( C:\Windows\Microsoft.NET\Framework64\v4.0.30319
  • 协议缓冲区2.5.0
  • Zlib和Zlib / include(以便它可以访问zlib.h)
  • FindBugs的
  • Maven的

我也设置了Platform=x64 ,不用说,我的JAVA_HOMEM2_HOME也都设置好了。

我在Windows SDK 7.1命令提示符下运行mvn package -Pdist,native-win,docs -DskipTests -Dtar

我早些时候有Gnuwin32这给了我下面的错误,但移动到Cygwin/bin怀疑Gnuwin32可能是错误的,但我仍然得到下面的错误。 我该怎么办?

这里是错误日志,

 [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.2:exec (com pile-ms-native-dll) on project hadoop-common: Command execution failed. Process exited with an error: 1(Exit value: 1) -> [Help 1] org.apache.maven.lifecycle.LifecycleExecutionException: Failed to execute goal o rg.codehaus.mojo:exec-maven-plugin:1.2:exec (compile-ms-native-dll) on project h adoop-common: Command execution failed. at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor .java:216) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor .java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor .java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProje ct(LifecycleModuleBuilder.java:116) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProje ct(LifecycleModuleBuilder.java:80) at org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThre adedBuilder.build(SingleThreadedBuilder.java:51) at org.apache.maven.lifecycle.internal.LifecycleStarter.execute(Lifecycl eStarter.java:120) at org.apache.maven.DefaultMaven.doExecute(DefaultMaven.java:347) at org.apache.maven.DefaultMaven.execute(DefaultMaven.java:154) at org.apache.maven.cli.MavenCli.execute(MavenCli.java:582) at org.apache.maven.cli.MavenCli.doMain(MavenCli.java:214) at org.apache.maven.cli.MavenCli.main(MavenCli.java:158) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl. java:57) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAcces sorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:606) at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced(Laun cher.java:289) at org.codehaus.plexus.classworlds.launcher.Launcher.launch(Launcher.jav a:229) at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode(La uncher.java:415) at org.codehaus.plexus.classworlds.launcher.Launcher.main(Launcher.java: 356) Caused by: org.apache.maven.plugin.MojoExecutionException: Command execution fai led. at org.codehaus.mojo.exec.ExecMojo.execute(ExecMojo.java:345) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(Default BuildPluginManager.java:132) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor .java:208) ... 19 more 

只有在hadoop-common ,它是失败的。 这里有一些可能有用的日志,我只是无法弄清楚它是什么,

 [INFO] [INFO] --- exec-maven-plugin:1.2:exec (compile-ms-native-dll) @ hadoop-common -- - [DEBUG] Configuring mojo org.codehaus.mojo:exec-maven-plugin:1.2:exec from plugi n realm ClassRealm[plugin>org.codehaus.mojo:exec-maven-plugin:1.2, parent: sun.m isc.Launcher$AppClassLoader@1e4ad14c] [DEBUG] Configuring mojo 'org.codehaus.mojo:exec-maven-plugin:1.2:exec' with bas ic configurator --> [DEBUG] (f) arguments = [C:\hdc\hadoop-common-project\hadoop-common/src/main/n ative/native.sln, /nologo, /p:Configuration=Release, /p:OutDir=C:\hdc\hadoop-com mon-project\hadoop-common\target/bin/, /p:CustomSnappyPrefix=, /p:CustomSnappyLi b=, /p:CustomSnappyInclude=, /p:RequireSnappy=false] [DEBUG] (f) basedir = C:\hdc\hadoop-common-project\hadoop-common [DEBUG] (f) classpathScope = runtime [DEBUG] (f) executable = msbuild [DEBUG] (f) longClasspath = false [DEBUG] (f) project = MavenProject: org.apache.hadoop:hadoop-common:2.5.2 @ C: \hdc\hadoop-common-project\hadoop-common\pom.xml [DEBUG] (f) session = org.apache.maven.execution.MavenSession@1e54fd3d [DEBUG] (f) skip = false [DEBUG] -- end configuration -- [DEBUG] Executing command line: msbuild C:\hdc\hadoop-common-project\hadoop-comm on/src/main/native/native.sln /nologo /p:Configuration=Release /p:OutDir=C:\hdc\ hadoop-common-project\hadoop-common\target/bin/ /p:CustomSnappyPrefix= /p:Custom SnappyLib= /p:CustomSnappyInclude= /p:RequireSnappy=false Building the projects in this solution one at a time. To enable parallel build, please add the "/m" switch. Build started 23-11-2014 00:07:43. Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.sln" on node 1 (default targets). ValidateSolutionConfiguration: Building solution configuration "Release|x64". Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.sln" (1) is building "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\nati ve.vcxproj" (2) on node 1 (default targets). InitializeBuildStatus: Touching "..\..\..\target\native\Release\native.unsuccessfulbuild". ClCompile: C:\Program Files (x86)\Microsoft Visual Studio 10.0\VC\bin\amd64\CL.exe /c /I. .\winutils\include /I..\..\..\target\native\javah /IC:\Java\jdk1.7.0_71\include /IC:\Java\jdk1.7.0_71\include\win32 /I.\src /Zi /nologo /W3 /WX- /O2 /Oi /GL /D WIN32 /D NDEBUG /D _WINDOWS /D _USRDLL /D NATIVE_EXPORTS /D _WINDLL /D _UNICODE /D UNICODE /Gm- /EHsc /MD /GS /Gy /fp:precise /Zc:wchar_t /Zc:forScope /Fo"..\.. \..\target\native\Release\\" /Fd"..\..\..\target\native\Release\vcWindows7.1SDK. pdb" /Gd /TC /wd4244 /errorReport:queue src\org\apache\hadoop\io\compress\zlib\Z libCompressor.c src\org\apache\hadoop\io\compress\zlib\ZlibDecompressor.c ZlibCompressor.c c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop \io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: C annot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-comm on-project\hadoop-common\src\main\native\native.vcxproj] ZlibDecompressor.c c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hadoop \io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: C annot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-comm on-project\hadoop-common\src\main\native\native.vcxproj] Done Building Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\nativ e\native.vcxproj" (default targets) -- FAILED. Done Building Project "C:\hdc\hadoop-common-project\hadoop-common\src\main\nativ e\native.sln" (default targets) -- FAILED. Build FAILED. "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.sln" (default target) (1) -> "C:\hdc\hadoop-common-project\hadoop-common\src\main\native\native.vcxproj" (def ault target) (2) -> (ClCompile target) -> c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hado op\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-co mmon-project\hadoop-common\src\main\native\native.vcxproj] c:\hdc\hadoop-common-project\hadoop-common\src\main\native\src\org\apache\hado op\io\compress\zlib\org_apache_hadoop_io_compress_zlib.h(36): fatal error C1083: Cannot open include file: 'zlib.h': No such file or directory [C:\hdc\hadoop-co mmon-project\hadoop-common\src\main\native\native.vcxproj] 0 Warning(s) 2 Error(s) 

从日志中,很显然你错过了zlib.h你是否将env var ZLIB_HOME设置为包含zlib.h的目录? 请注意,在PATH中放置zlib.h目录是错误的,没有必要,只需要zlib bin目录,如在建筑指南中所述https://svn.apache.org/viewvc/hadoop/common/branches/branch-2/BUILDING .TXT?视图=标记

另外,zlib.h需要在Windows中不存在的头文件,因此您需要下载这些头文件并将其放在与zlib.h相同的文件夹中。 有关更多详细信息,请参阅是否有Windows for unistd.h的替代品(Visual C)? 得到unistd.h,并https://gist.github.com/ashelly/7776712得到getopt.h

对于Win32上的那些人(像我),你可能会面临更多的麻烦。 关键的解决方案应该是编辑winutils和本地包的.sln和.vcxprj文件,以便它们与Win32平台兼容。

我亲自使用了命令mvn clean package -Pdist,native-win -DskipTests -Dtar -Dmaven.javadoc.skip=true在Windows 8上成功构建2.4.1版本。我有许多不同的问题,但是不要记住这是其中之一。 但它似乎抱怨zlib。