Julia crashes using [email protected]

[sebastianangerhausen]

After updating Gurobi_jll from version 11.0.2 to 11.0.3, the Julia instance crashes every time in a way that I can only quickly take a screenshot until the REPL is closed automatically. This is the information I could retain:

I believe, the responsible line is Solver.jl:82, which contains

set_optimizer(jump_model, () -> Gurobi.Optimizer(solver_env))

where solver_env is the previously created solver environment.

When I updated the Gurobi.jl package, I got the following warnings:

WARNING: redefinition of constant Gurobi_jll.PATH. This may fail, cause incorrect answers, or produce other errors.
WARNING: redefinition of constant Gurobi_jll.LIBPATH. This may fail, cause incorrect answers, or produce other errors.
WARNING: redefinition of constant Gurobi_jll.PATH_list. This may fail, cause incorrect answers, or produce other errors.
WARNING: redefinition of constant Gurobi_jll.LIBPATH_list. This may fail, cause incorrect answers, or produce other errors.
┌ Error: 2024-10-31 17:51:31 Error watching manifest
│   exception =
│    cannot set type for global Gurobi_jll.best_wrapper. It already has a value or is already set to a different type.
│    Stacktrace:
│     [1] top-level scope
│       @ C:\Users\angerhse\.julia\packages\JLLWrappers\jXOYx\src\toplevel_generators.jl:119
│    Revise evaluation error at C:\Users\angerhse\.julia\packages\JLLWrappers\jXOYx\src\toplevel_generators.jl:119
│
│    Stacktrace:
│     [1] methods_by_execution!(recurse::Any, methodinfo::Revise.CodeTrackingMethodInfo, docexprs::Dict{Module, Vector{Expr}}, mod::Module, ex::Expr; mode::Symbol, disablebp::Bool, always_rethrow::Bool, kwargs::@Kwargs{})
│       @ Revise C:\Users\angerhse\.julia\packages\Revise\uvGMC\src\lowered.jl:281
└ @ Revise C:\Users\angerhse\.julia\packages\Revise\uvGMC\src\pkgs.jl:495

I am using a Windows computer, so the x86_64-w64-mingw32 wrapper should be the one in question.

@odow If I remember correctly, you don't use Windows. So please let me know, if I can provide any more details helping to solve the issue.

[odow]

This is very weird. You should not have been able to trigger the WARNING: redefinition of constant.

How did you update Gurobi_jll?

Try:

] add Gurobi Gurobi_jll
] precompile Gurobi_jll
] precompile Gurobi

Close Julia, then restart Julia.

[odow]

I think this is because you had Revise loaded when you updated the package, and it's managed to get things into a weird state. This doesn't seem like an issue in Gurobi_jll.jl.

[odow]

Any update? If not, I will close this issue. I don't think it's a problem with Gurobi_jll.

[sebastianangerhausen]

I still encounter the issue, even if Revise is not loaded. I tried to build a MWE:

module TestGurobi

using Gurobi
using JuMP

function test_gurobi()
    model = Model()
    env = Gurobi.Env(started = false)
    Gurobi.GRBstartenv(env)
    set_optimizer(model, () -> Gurobi.Optimizer(env))

    finalize(backend(model).optimizer.model)
    finalize(env)
end

end

I put this into the file TestGurobi.jl in the newly generated package TestGurobi. Then I tried two different approaches to add the dependencies:

1. add Gurobi
2. add Gurobi Gurobi_jll also with the proposed precompilation steps

After that, executing using TestGurobi and TestGurobi.test_gurobi() leads most of the times to the following error:

ERROR: Gurobi Error 10009:
Stacktrace:
  [1] _check_ret
    @ C:\Users\angerhse\.julia\packages\Gurobi\lpV4m\src\MOI_wrapper\MOI_wrapper.jl:376 [inlined]
  [2] empty!(model::Gurobi.Optimizer)
    @ Gurobi C:\Users\angerhse\.julia\packages\Gurobi\lpV4m\src\MOI_wrapper\MOI_wrapper.jl:444
  [3] Gurobi.Optimizer(env::Gurobi.Env; enable_interrupts::Bool)
    @ Gurobi C:\Users\angerhse\.julia\packages\Gurobi\lpV4m\src\MOI_wrapper\MOI_wrapper.jl:349
  [4] Optimizer
    @ C:\Users\angerhse\.julia\packages\Gurobi\lpV4m\src\MOI_wrapper\MOI_wrapper.jl:326 [inlined]
  [5] (::TestGurobi.var"#1#2"{Gurobi.Env})()
    @ TestGurobi D:\DevOps\TestGurobi\src\TestGurobi.jl:10
  [6] _instantiate_and_check(optimizer_constructor::Any)
    @ MathOptInterface C:\Users\angerhse\.julia\packages\MathOptInterface\gLl4d\src\instantiate.jl:94
  [7] instantiate(optimizer_constructor::Any; with_bridge_type::Type{Float64}, with_cache_type::Nothing)
    @ MathOptInterface C:\Users\angerhse\.julia\packages\MathOptInterface\gLl4d\src\instantiate.jl:175
  [8] set_optimizer(model::JuMP.Model, optimizer_constructor::Any; add_bridges::Bool)
    @ JuMP C:\Users\angerhse\.julia\packages\JuMP\FEKLB\src\optimizer_interface.jl:493
  [9] set_optimizer
    @ C:\Users\angerhse\.julia\packages\JuMP\FEKLB\src\optimizer_interface.jl:486 [inlined]
 [10] test_gurobi()
    @ TestGurobi D:\DevOps\TestGurobi\src\TestGurobi.jl:10
 [11] top-level scope
    @ REPL[5]:1

However, after repeating the execution a few times, I always end up with a crashed Julia instance as described in the original post.

Executing all of the above steps with add Gurobi [email protected] instead of add Gurobi Gurobi_jll works as expected.

[sebastianangerhausen]

I also looked at the manifest of the working approach and compared it to the failing one. The difference is really just in the version of Gurobi_jll:

[odow]

Error 10009 is no license: https://www.gurobi.com/documentation/current/refman/error_codes.html

You need to get a license for Gurobi 12 if you want to use Gurobi_jll@12.

[sebastianangerhausen]

I get the same error using [email protected], so apparently it does not result from a wrong Gurobi version in the license file. Moreover, since I am using a compute server, I usually do not need to update the local license file if the Gurobi version of the server changes. Is there a way to get a more detailed error description? Apparently, there are different forms of error 10009 (e.g. "Unable to open Gurobi license file", "License expired", or "HostID mismatch").

[sebastianangerhausen]

The problem might just be that our compute server is still running on version 11.0.2 while I am trying to use the binary of version 11.0.3 or even 12.0.0. It was not clear to me that there is no backwards compatability in this direction (whereas I can update the server and keep using the older binary). Sorry!

[simonbowly]

Is there a way to get a more detailed error description?

In your test function you called a lower level function directly without checking the error code or message:

Gurobi.GRBstartenv(env)

Either of the following would give you the full error:

using JuMP, Gurobi
model = Model(Gurobi.Optimizer)

using Gurobi
env = Gurobi.Env()

These would both show the following error message if the issue is that the compute server is not up to date:

LoadError: Gurobi Error 10030: No compatible runtime available for version 12.0.0

[sebastianangerhausen]

This is a good tip! It appears that the error code shown by my original code is always 10009, even if there is no problem with the license file. Using your proposed method works well. It turned out that the actual issue after updating the compute server was that I didn't wait long enough for the server to obtain a valid license from the WLS service.

However, the reason for my code choice was that I need to modify some parameters before starting the environment:

env = Gurobi.Env(started = false)
Gurobi.GRBsetstrparam(env, "CSAppName", "some name")
Gurobi.GRBsetstrparam(env, "CSManager", "some url")
Gurobi.GRBsetintparam(env, "CSPriority", 10)
Gurobi.GRBstartenv(env)

Is there an other way to do that without using the lower level function Gurobi.GRBstartenv(env)?

[simonbowly]

Is there an other way to do that without using the lower level function Gurobi.GRBstartenv(env)?

I don't think Gurobi.jl has another way to pass configuration variables before starting an environment. There is a specific constructor for compute servers, but not for the cluster manager. The commonly used approach would be to have these parameters in a license file (gurobi.lic) instead.

To set the parameters in the code rather than a license file: you can still take the approach you've used, but since you're calling the C API functions directly you must check for the error code and error message. These checks are done in the Gurobi.jl wrapper using a function _check_ret that isn't exposed, so I suggest adding that to your code:

using JuMP, Gurobi

model = Model()

function _check_ret(env, ret::Cint)
    if ret != 0
        msg = unsafe_string(GRBgeterrormsg(env))
        throw(ErrorException("Gurobi Error $(ret): $(msg)"))
    end
    return
end

env = Gurobi.Env(started = false)

ret = Gurobi.GRBsetstrparam(env, "CSAppName", "some name")
_check_ret(env, ret)

ret = Gurobi.GRBsetstrparam(env, "CSManager", "some url")
_check_ret(env, ret)

ret = Gurobi.GRBsetintparam(env, "CSPriority", 10)
_check_ret(env, ret)

ret = Gurobi.GRBstartenv(env)
_check_ret(env, ret)

set_optimizer(model, () -> Gurobi.Optimizer(env))

optimize!(model)

finalize(backend(model).optimizer.model)
finalize(env)

The parameters are obviously wrong, but you'll get a valid error message this way:

ERROR: LoadError: Gurobi Error 10022: URL rejected: Malformed input to a URL function (code 3, command POST some url/api/v1/auth/login)

[simonbowly]

I opened jump-dev/Gurobi.jl#588 to propose a supported approach for setting these configuration parameters.

[odow]

Closing because the original issue seems resolved (please comment if not, and I will re-open), and it isn't a bug in Gurobi_jll. The Env parameters can be discussed in jump-dev/Gurobi.jl#588