Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

KNITRO not found on macOS #225

Closed
dpo opened this issue Jul 14, 2022 · 12 comments
Closed

KNITRO not found on macOS #225

dpo opened this issue Jul 14, 2022 · 12 comments

Comments

@dpo
Copy link
Contributor

dpo commented Jul 14, 2022

I'm on macOS M1 and for some reason, KNITRO.jl isn't finding my installation of KNITRO:

(@v1.7) pkg> activate test-lin-cons/
  Activating project at `~/dev/JSO/NCL.jl/test-lin-cons`

(test-lin-cons) pkg> build KNITRO
    Building KNITRO → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/f880c76ceb1b5b776f4dd57b01c5b2d1954a56e6/build.log`
Precompiling project...
  2 dependencies successfully precompiled in 7 seconds (51 already precompiled)

julia> using KNITRO
[ Info: Precompiling KNITRO [67920dd8-b58e-52a8-8622-53c4cffbe346]

julia> KNITRO.has_knitro()
false

julia> ENV["KNITRODIR"]
"/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1"

julia> ENV["LD_LIBRARY_PATH"]
"/usr/local/lib:/usr/local/lib::/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib"

julia> ENV["DYLD_LIBRARY_PATH"]
"/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib:"

(test-lin-cons) pkg> st
      Status `~/dev/JSO/NCL.jl/test-lin-cons/Project.toml`
  [54578032] ADNLPModels v0.3.3
  [4076af6c] JuMP v1.1.1
  [67920dd8] KNITRO v0.13.0
  [792afdf1] NLPModelsJuMP v0.11.0
  [bec4dd0d] NLPModelsKnitro v0.5.7

I don't think LD_LIBRARY_PATH is particularly relevant on macOS. I used to have to set DYLD_LIBRARY_PATH instead, but that doesn't seem to do it either.

I can certainly use the knitroampl binary from the shell, so my license is ok. The build.log file is empty.

If I try to run the unit tests, I get messages of the form

Instantiation Knitro C interface: Error During Test at /Users/dpo/.julia/packages/KNITRO/6cfYO/test/C_wrapper.jl:37
  Got exception outside of a @test
  could not load library ""
  dlopen(.dylib, 0x0001): tried: '/.dylib' (no such file), '/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib/.dylib' (no such file), '/Applications/Julia-1.7.app/Contents/Resources/julia/lib/julia/.dylib' (no such file), '/Applications/Julia-1.7.app/Contents/Resources/julia/bin/../lib/.dylib' (no such file), '.dylib' (no such file), '/usr/local/lib/.dylib' (no such file), '/usr/lib/.dylib' (no such file), '/.dylib' (no such file), '/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib/.dylib' (no such file), '/Users/dpo/.julia/packages/KNITRO/6cfYO/test/.dylib' (no such file)

Any ideas?
@frapac
Copy link
Collaborator

frapac commented Jul 14, 2022

That's indeed unexpected. Pkg.build looks for a shared library both in $KNITRODIR/lib and in LD_LIBRARY_PATH. Your KNITRODIR looks correct, so it should be able to find the library on your computer. What's the output of the following command?

ls $KNITRODIR/lib

@dpo
Copy link
Contributor Author

dpo commented Jul 14, 2022 

$ ls -l $KNITRODIR/lib
total 25200
lrwxr-xr-x@ 1 dpo  staff    15B Feb 10 13:29 libknitro.a -> libknitro1301.a
lrwxr-xr-x@ 1 dpo  staff    19B Feb 10 13:29 libknitro.dylib -> libknitro1301.dylib
lrwxr-xr-x@ 1 dpo  staff    19B Feb 10 13:29 libknitro.dylib.13 -> libknitro1301.dylib
lrwxr-xr-x@ 1 dpo  staff    19B Feb 10 13:29 libknitro.dylib.13.0 -> libknitro1301.dylib
lrwxr-xr-x@ 1 dpo  staff    19B Feb 10 13:29 libknitro.dylib.13.0.1 -> libknitro1301.dylib
-rw-r--r--@ 1 dpo  staff   6.6M Feb 10 13:25 libknitro1301.a
-rwxr-xr-x@ 1 dpo  staff   5.1M Feb 10 13:25 libknitro1301.dylib
-rw-r--r--@ 1 dpo  staff   647K Feb  1 13:26 libomp.dylib

@frapac
Copy link
Collaborator

frapac commented Jul 14, 2022

ok. And what's the output of the following command in Julia?

using Libdl
knitro_lib = joinpath(ENV["KNITRODIR"], "lib", "libknitro.dylib")
Libdl.dlopen_e(knitro_lib)

@dpo
Copy link
Contributor Author

dpo commented Jul 14, 2022 •
edited
Loading 

julia> using Libdl

julia> knitro_lib = joinpath(ENV["KNITRODIR"], "lib", "libknitro.dylib")
"/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib/libknitro.dylib"

julia> Libdl.dlopen_e(knitro_lib)
Ptr{Nothing} @0x0000000000000000

julia> Libdl.dlopen(knitro_lib)
ERROR: could not load library "/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib/libknitro.dylib"
dlopen(/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib/libknitro.dylib, 0x0001): tried: '/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib/libknitro.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/libknitro.dylib' (no such file), '/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib/libknitro.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib/libknitro1301.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64')), '/libknitro1301.dylib' (no such file), '/Users/dpo/local/solvers/knitro/knitro-13.0.1-MacOS-M1/lib/libknitro1301.dylib' (mach-o file, but is an incompatible architecture (have 'arm64', need 'x86_64'))
Stacktrace:
 [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
   @ Base.Libc.Libdl ./libdl.jl:117
 [2] dlopen (repeats 2 times)
   @ ./libdl.jl:117 [inlined]
 [3] top-level scope
   @ REPL[4]:1

Is this due to the fact I have to run Julia for x86_64 on the M1, yet the Knitro lib is for M1? I just tried with Ipopt and it seems to be working... though the unit tests hang midway... 🤔

@frapac
Copy link
Collaborator

frapac commented Jul 14, 2022

Ipopt standalone or Ipopt.jl? It looks like Ipopt.jl has also its bunch of issues: jump-dev/Ipopt.jl#315

So apparently for Julia x86_64 is not able to load the Knitro lib for M1, as you suggested. Would it be possible to use Julia 1.8 with the M1 build? https://julialang-s3.julialang.org/bin/mac/aarch64/1.8/julia-1.8.0-rc1-macaarch64.dmg

@dpo
Copy link
Contributor Author

dpo commented Jul 14, 2022

Would the Knitro 86_64 run on the M1? Is it possible to install it alongside the M1 lib?

@frapac
Copy link
Collaborator

frapac commented Jul 14, 2022 •
edited
Loading

I think you can download Knitro x86_64 on the M1. Not sure it will work out of the box (the x86_64 build is compiled with MKL). Another possibility is to use Knitro x86_64 with rosetta.

@odow
Copy link
Member

odow commented Aug 17, 2022

Is this resolved? You should be able to run KNITRO x86_64 on the M1 with Julia x86_64, Or KNITRO arm64 with Julia arm64.

I have an M1, but not a KNITRO license, so I can't test unfortunately.

x-ref: jump-dev/JuMP.jl#2971. If you previously tried on Julia 1.7, try with the latest 1.8-rc4. It fixes a lot of issues.

@dpo
Copy link
Contributor Author

dpo commented Aug 18, 2022

I'm still getting the same errors as above after installing KNITRO 13.1 for x86_64 and running it in Julia 1.7.3 for x86_64. I'll test Julia 1.8-rc4 as soon as it appears on juliaup. I can run knitroampl just fine from the command line.

@dpo
Copy link
Contributor Author

dpo commented Aug 18, 2022

Upgrading juliaup gave access to 1.8-rc4. The results are more promising but a bunch of tests fail. KNITRO.has_knitro() finally returns true. The complete output is here: https://gist.github.com/dpo/2df7b64b32855619206b608517050413

@odow
Copy link
Member

odow commented Aug 18, 2022

The test failures are #224 and #228, but it looks like the M1 is working fine.

@odow
Copy link
Member

odow commented Aug 24, 2022

Closing because tests are passing (although we need to fix a bunch of excluded ones): #230

@odow odow closed this as completed Aug 24, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@dpo @odow @frapac