yara @1.7: update to 3.3.0

[jul_bsd@…]

Malware identification and classification tool

use a post 3,1,0 as discussed with author ​https://github.com/plusvic/yara/issues/168

• keep existing v1 as subport
• current 3.1.0
• subport devel for latest
• universal ok (test w 3.1 only)

destroot ok for 1.7, 3.1.0, devel

[ryandesign (Ryan Carsten Schmidt)]

Thanks. Some observations:

• When using the github portgroup, the portgroup is expected to take care of all needs regarding master_sites, distname, worksrcdir; you're not expected to need to set them manually.
• The dependency on libtool should be written in port: style, not bin: style; OS X does not come with /usr/bin/glibtool and we would not want a rogue /usr/bin/glibtool installed by the user to satisfy the dependency.
• system's -W argument should be used instead of running a cd command within it.
• The python28 variant should be deleted because no such version of python exists.
• The python27 variant should only be enabled by default it the user has not already selected another python variant; see PortfileRecipes#default_variants
• The python variants should be marked as conflicting with one another.
• For the ${name}-devel subport, is fetching from git truly necessary? Fetching from a distfile is preferred, and is the github portgroup's default operating mode.
• The livecheck looks like it would only be applicable to one of the subports. (Which one?) Should the other subports get their own livecheck? Note that the github portgroup includes a working default livecheck definition.

[jul_bsd@…]

fixed comments distname is kept separate as we are not using a release. there were osx mandatory bugfix after-release. so for now 3.1.0 and devel are the same but it will evolve over time. same for worksrcdir as it's not a full release.

there are a problem with python bindings as they have a setup.py which is not handle by configure. It's still the same archive. What is the proper way to handle it? subport, variant, different Portfile, with python portgroup?

[mf2k (Frank Schima)]

Please attach an updated patch of the Portfile.

[jul_bsd@…]

removed python variant for a separate port py-yara

[jul_bsd@…]

• subport 3.1+, update latest devel
• current is 3.2
• compile issue reported to upstream

[a.schuster@…]

Sorry for the wait - attached is the update to the lastest version of YARA, v 3.3.0.

This update adds variants to enable the new plugins/modules of YARA: cuckoo, hash, magic, and math. Note, the modules pe and elf are always enabled, as per the YARA source.

Also, this update finally adds the python bindings as a subport. The unified python PortGroup did not work in that context as I expected. I tried to resemble its behaviour as closely as possible, though. The port will now run the test case that comes with the source distribution.

Another option to install the python bindings would be to provide a separate Portfile, as jul_bsd has proposed. This clearly would ease the task of creating the Portfile, because the python PortGroup and all its magic would be available. On the other hand, this would require maintenance of two Portfiles, that both rely on the same source tarball. As I am still learning how to handle Portfiles and don't have commit privileges, maintaing a single file seems the easier and more stable approach to me. Thus no desync'ed versions of library and bindings can exist in the port repository.

I'm attaching the diff and, for the sake of clarity, also the complete Portfile.

[a.schuster@…]

diff of Portfile for v3.3.0 against v1.7

[a.schuster@…]

complete Portfile for v3.3.0 for clarity

[jul_bsd@…]

Thanks a lot Andreas

I merged the two Portfile to keep devel and old v1. Also, on my box which I reinstall fully recently (macos 10.10 / darwin 14), it didn't compile :( same for latest yara-devel. yara1 is ok

lot of

yarac.c:243:34: error: use of undeclared identifier 'compiler'
  result = yr_compiler_get_rules(compiler, &rules);
                                 ^
yarac.c:251:12: warning: implicit declaration of function 'yr_rules_save' is invalid in C99 [-Wimplicit-function-declaration]
  result = yr_rules_save(rules, argv[argc - 1]);
           ^
yarac.c:251:26: error: use of undeclared identifier 'rules'
  result = yr_rules_save(rules, argv[argc - 1]);
                         ^
yarac.c:263:7: error: use of undeclared identifier 'compiler'
  if (compiler != NULL)
      ^
yarac.c:264:5: warning: implicit declaration of function 'yr_compiler_destroy' is invalid in C99 [-Wimplicit-function-declaration]
    yr_compiler_destroy(compiler);

I joined the log file and diff

I removed python bindings for 26/31/32 as on other ports I was said macports minimum is now 2.7 or 3.3

[jul_bsd@…]

Discussed with andreas. build conflict with existing yara1. build fine if nothing existing. So it seems conflict doesn't apply to build phase.

tried to force reordering search on cppflags or cflags but doesn't seem to help

:info:build     mv -f $depbase.Tpo $depbase.Plo
:info:build libtool: compile:  /usr/bin/clang -DHAVE_CONFIG_H -I. -I.. -I/opt/local/include -O3 -Wall -Wno-deprecated-declarations -std=gnu99 -I./include -pipe -Os -I./include -I/opt/local/include -arch x86_64 -D_THREAD_SAFE -MT exec.lo -MD -MP -MF .deps/exec.Tpo -c exec.c  -fno-common -DPIC -o .libs/exec.o
:info:build In file included from exec.c:33:
:info:build /opt/local/include/yara.h:80:9: warning: 'ERROR_UNDEFINED_STRING' macro redefined

any way to ensure yara1 is not installed when doing the build? when committed, it shouldn't be a problem as for most people, the correct archive will be fetched and intall directly.

yara and py27 install fine if no yara1. seems ready for commit. other comments?

[a.schuster@…]

Thanks, I think we're approaching a solution.

YARA 1.x is outdated and no longer supported. However, a few people (and applications) still require it. Therefore I propose to add a new port, named yara1, which provides YARA v1.7. We could add pcre/re2 as variants and the proper Python bindings as a subport. A single commit is required; the port should be stable for a fairly long time.

This port (yara) would track the stable release only, with variants for modules and a subport for Python bindings. Commits are required for new stable releases of YARA.

Now on to the bleeding edge development version. It doubt it is feasible to release a patch whenever YARA gets updated, as this may happen many times a week. It would impose a high workload on those kind guys with commit privilege. How do other projects deal with situation? Do we really need a yara-devel port? After all, starting from the current Portfile and tweaking it to your needs shouldn't be too hard for a serious developer.

Or, we could provide a yara-devel Portfile as a template. But then we need to be very clear about it being only a template and not following HEAD on github.

All the ports (yara1, yara, and yara-devel) would be mutually exclusive.

[jul_bsd@…]

I don't think there is a clear policy for one product to use only subport or different Portfile. I think the first one is more coherent but depends. the conflict settings will not be changed by the fact there is one or multiple Portfile AFAIK and as seen above, it only applies to install not build conflict.

yara1 subport should be fully operational with re-subport. I'm adding a warning of EOL. all of that shouldn't change much once we catch latest release.

On devel, it's more a pointer for maintainers or more advanced people, to test most recent stuff inside macports but probably to be removed or commented for commit. (prefering second). currently not compiling and we don't want to commit that. Clearly the goal is not to track every commit. I think, it's more useful for projects which have long release cycle or no release at all.

[kurthindenburg (Kurt Hindenburg)]

Cc Me!

[kurthindenburg (Kurt Hindenburg)]

I've been testing this - I updated to handle 3.4.0 - ignoring yara1 would make the Portfile easier to handle - is there any reason to not include the non-python variants by default?

[kurthindenburg (Kurt Hindenburg)]

I went ahead and committed what I have r145484. If you can, try it out and let me know of any issues. The only port that depends on it is volatility which I'll look at shortly.

[nihilus (Markus Gothe)]

Actually it won't build on:

zarathustra:lltdscan nietzsche$ uname -a
Darwin zarathustra.local 13.4.0 Darwin Kernel Version 13.4.0: Wed Mar 18 16:20:14 PDT 2015; root:xnu-2422.115.14~1/RELEASE_X86_64 x86_64

it bails with following errors:

make[2]: Entering directory `/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_security_yara/yara/work/yara-3.4.0'
depbase=`echo args.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
	/usr/bin/clang -DHAVE_CONFIG_H -I.   -I/opt/local/include -O3 -std=gnu99 -Wall -I./libyara/include -pipe -Os -std=gnu89 -arch x86_64 -D_THREAD_SAFE  -MT args.o -MD -MP -MF $depbase.Tpo -c -o args.o args.c &&\
	mv -f $depbase.Tpo $depbase.Po
depbase=`echo threading.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
	/usr/bin/clang -DHAVE_CONFIG_H -I.   -I/opt/local/include -O3 -std=gnu99 -Wall -I./libyara/include -pipe -Os -std=gnu89 -arch x86_64 -D_THREAD_SAFE  -MT threading.o -MD -MP -MF $depbase.Tpo -c -o threading.o threading.c &&\
	mv -f $depbase.Tpo $depbase.Po
depbase=`echo yara.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
	/usr/bin/clang -DHAVE_CONFIG_H -I.   -I/opt/local/include -O3 -std=gnu99 -Wall -I./libyara/include -pipe -Os -std=gnu89 -arch x86_64 -D_THREAD_SAFE  -MT yara.o -MD -MP -MF $depbase.Tpo -c -o yara.o yara.c &&\
	mv -f $depbase.Tpo $depbase.Po
depbase=`echo yarac.o | sed 's|[^/]*$|.deps/&|;s|\.o$||'`;\
	/usr/bin/clang -DHAVE_CONFIG_H -I.   -I/opt/local/include -O3 -std=gnu99 -Wall -I./libyara/include -pipe -Os -std=gnu89 -arch x86_64 -D_THREAD_SAFE  -MT yarac.o -MD -MP -MF $depbase.Tpo -c -o yarac.o yarac.c &&\
	mv -f $depbase.Tpo $depbase.Po
yarac.c:94:22: error: use of undeclared identifier 'YARA_ERROR_LEVEL_ERROR'
  if (error_level == YARA_ERROR_LEVEL_ERROR)
                     ^
yarac.c:107:5: error: unknown type name 'YR_COMPILER'
    YR_COMPILER* compiler)
    ^
yarac.c:130:7: warning: implicit declaration of function 'yr_compiler_define_integer_variable' [-Wimplicit-function-declaration]
      yr_compiler_define_integer_variable(
      ^
yarac.c:137:7: warning: implicit declaration of function 'yr_compiler_define_boolean_variable' [-Wimplicit-function-declaration]
      yr_compiler_define_boolean_variable(
      ^
yarac.c:144:7: warning: implicit declaration of function 'yr_compiler_define_string_variable' [-Wimplicit-function-declaration]
      yr_compiler_define_string_variable(
      ^
yarac.c:125:11: warning: unused variable 'identifier' [-Wunused-variable]
    char* identifier = ext_vars[i];
          ^
yarac.c:162:3: error: use of undeclared identifier 'YR_COMPILER'
  YR_COMPILER* compiler = NULL;

[a.schuster@…]

In file included from exec.c:34:
/opt/local/include/yara.h:39:9: warning: 'LEX_BUF_SIZE' macro redefined
#define LEX_BUF_SIZE                            1024
        ^

It includes yara.h of the already installed version, not the new one in the temporary build directory.

Workaround: uninstall the old version first, then build the new one.

Or, even better, modify the include search order in the Makefile. In other words, -I./libyara/include should come first, then -I/opt/local/include. Unfortunately I have zero knowledge about the Makefile generation process.

Help, anyone? Thanks!

[kurthindenburg (Kurt Hindenburg)]

Please sync and try again if anyone is still having this issue r145541