Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support declarative HTTP clients #31337

Open
mhalbritter opened this issue Jun 10, 2022 · 31 comments · May be fixed by #42003
Open

Support declarative HTTP clients #31337

mhalbritter opened this issue Jun 10, 2022 · 31 comments · May be fixed by #42003
Labels
type: enhancement A general enhancement
Milestone

Comments

@mhalbritter
Copy link
Contributor

Spring Framework 6.0 introduces declarative HTTP clients. We should add some auto-configuration in Boot which supplies for example the HttpServiceProxyFactory.

We could even think about an annotation with which the HTTP interface can be annotated and then directly injected into a consumer (like @FeignClient).

@mhalbritter mhalbritter added type: enhancement A general enhancement status: pending-design-work Needs design work before any code can be developed labels Jun 10, 2022
@mhalbritter mhalbritter added this to the 3.x milestone Jun 10, 2022
@livk-cloud
Copy link

livk-cloud commented Jun 11, 2022

I think this one can try to inject IOC by myself
For example I do

public class HttpServiceFactory implements BeanFactoryAware, ImportBeanDefinitionRegistrar, ResourceLoaderAware {

	private final HttpServiceProxyFactory proxyFactory;

	private BeanFactory beanFactory;

	private ResourceLoader resourceLoader;

	public HttpServiceFactory() {
		WebClient client = WebClient.builder().build();
		this.proxyFactory = HttpServiceProxyFactory.builder(new WebClientAdapter(client)).build();
	}

	@Override
	public void registerBeanDefinitions(@NonNull AnnotationMetadata importingClassMetadata,
			@NonNull BeanDefinitionRegistry registry) {
		List<String> packages = AutoConfigurationPackages.get(this.beanFactory);
		Set<Class<?>> typesAnnotatedClass = findByAnnotationType(HttpExchange.class, resourceLoader,
				packages.toArray(String[]::new));
		for (Class<?> exchangeClass : typesAnnotatedClass) {
			BeanName name = AnnotationUtils.getAnnotation(exchangeClass, BeanName.class);
			String beanName = name != null ? name.value()
					: CaseFormat.UPPER_CAMEL.to(CaseFormat.LOWER_CAMEL, exchangeClass.getSimpleName());
			registry.registerBeanDefinition(beanName, getBeanDefinition(exchangeClass));
		}
	}

	private <T> BeanDefinition getBeanDefinition(Class<T> exchangeClass) {
		return new RootBeanDefinition(exchangeClass, () -> proxyFactory.createClient(exchangeClass));
	}

	@Override
	public void setResourceLoader(@NonNull ResourceLoader resourceLoader) {
		this.resourceLoader = resourceLoader;
	}

	@Override
	public void setBeanFactory(@NonNull BeanFactory beanFactory) throws BeansException {
		this.beanFactory = beanFactory;
	}

public Set<Class<?>> findByAnnotationType(Class<? extends Annotation> annotationClass,
			ResourceLoader resourceLoader, String... packages) {
		Assert.notNull(annotationClass, "annotation not null");
		Set<Class<?>> classSet = new HashSet<>();
		if (packages == null || packages.length == 0) {
			return classSet;
		}
		ResourcePatternResolver resolver = ResourcePatternUtils.getResourcePatternResolver(resourceLoader);
		CachingMetadataReaderFactory metadataReaderFactory = new CachingMetadataReaderFactory(resourceLoader);
		try {
			for (String packageStr : packages) {
				packageStr = packageStr.replace(".", "/");
				Resource[] resources = resolver.getResources("classpath*:" + packageStr + "/**/*.class");
				for (Resource resource : resources) {
					MetadataReader metadataReader = metadataReaderFactory.getMetadataReader(resource);
					String className = metadataReader.getClassMetadata().getClassName();
					Class<?> clazz = Class.forName(className);
					if (AnnotationUtils.findAnnotation(clazz, annotationClass) != null) {
						classSet.add(clazz);
					}
				}
			}
		}
		catch (IOException | ClassNotFoundException e) {
			throw new RuntimeException(e);
		}
		return classSet;
	}

}

/**
 * Just used to set the BeanName
 */
@Target(ElementType.TYPE)
@Retention(RetentionPolicy.RUNTIME)
public @interface BeanName {

	String value();

}

What is your opinion on this

@spencergibb
Copy link
Member

spencergibb commented Jun 13, 2022

What would an additional annotation bring beyond @HttpMapping?

@rstoyanchev
Copy link
Contributor

The underlying client may need to be configured with different base URL, codecs, etc. That means detecting @HttpExchange annotated interfaces and declaring beans for them could become inflexible if it's based on the same underlying client setup.

Given an HttpServiceProxyFactory, it's trivial to create a proxy, so don't see a lot of gain from automating that. What I am thinking about though is that it should be possible for Boot to create and auto-configure one HttpServiceProxyFactory instance that can then be combined that with different client setups.

Currently HttpServiceProxyFactory takes HttpClientAdapter in the constructor, but we could make a change to allow it to be passed to an overloaded createClient(Class<?>, HttpClientAdapter) method. So you could inject the same HttpServiceProxyFactory anywhere, and either use it with the default client setup (based on WebClientCustomizer and WebClient.Builder), or optionally, also inject WebClient.Builder and use a client setup that deviates from the default.

@mhalbritter
Copy link
Contributor Author

I played around with it here.

The auto-configuration supplies a bean of type HttpServiceProxyFactory, which a user can then use to create the proxy from the interface. The base url for the client is settable via @HttpExchange and is not configured on the factory.

@rstoyanchev
Copy link
Contributor

Thanks, this helps me to move my thought process forward. I see now it is necessary to separate more formally the HttpServiceProxyFactory from the underlying client.

I've an experiment locally where HttpServiceProxyFactory expects the HttpClientAdapter to be passed in every time createClient is called. Separately, there is a WebClientServiceProxyFactory that is created with an HttpServiceProxyFactory and a WebClient and exposes a createClient with just the proxy interface.

The Boot auto-config could then declare a single HttpServiceProxyFactory bean, and applications would create any number of WebClientServiceProxyFactory beans, each delegating to the same HttpServiceProxyFactory and the WebClient configured for a specific remote.

@rstoyanchev
Copy link
Contributor

After a few different experiments, I think trying to have one HttpServiceProxyFactory for many client instances brings additional complexity with little gain. The easiest to understand model remains, one HttpServiceProxyFactory for one client. It's reasonably simple even without any help from Boot:

@Bean
HttpServiceProxyFactory httpServiceProxyFactory1(WebClient.Builder clientBuilder) {
	WebClient client = clientBuilder.baseUrl("http://host1.com").build();
	return new HttpServiceProxyFactory(new WebClientAdapter(client));
}

@Bean
HttpServiceProxyFactory httpServiceProxyFactory2(WebClient.Builder clientBuilder) {
	WebClient client = clientBuilder.baseUrl("http://host2.com").build();
	return new HttpServiceProxyFactory(new WebClientAdapter(client));
}

A couple of extra shortcuts on WebClientAdapter could make this a one-liner:

@Bean
HttpServiceProxyFactory httpServiceProxyFactory1(WebClient.Builder clientBuilder) {
	return WebClientAdapter.createProxyFactory(clientBuilder.baseUrl("http://host1.com"));
}

@Bean
HttpServiceProxyFactory httpServiceProxyFactory2(WebClient.Builder clientBuilder) {
	return WebClientAdapter.createProxyFactory(clientBuilder.baseUrl("http://host2.com"));
}

If we settle on the above as the expected configuration, then I think it's not essential to have any Boot auto-config to start, although some ideas may still come along. Perhaps, specifying baseUrl's in properties, which would allow Boot to create the above beans?

@wilkinsona
Copy link
Member

Thanks, Rossen.

Perhaps, specifying baseUrl's in properties, which would allow Boot to create the above beans?

We don't yet support specifying multiple property values to auto-configure multiple beans anywhere in Boot. It's something that we'd like to do, but it's a complex and wide-ranging topic. #15732 is tracking auto-configured multiple DataSources, for example.

Having discussed this today, we don't think there's anything to do in Boot at this time. We can revisit this if the picture changes for auto-configuring multiple beans.

@wilkinsona wilkinsona closed this as not planned Won't fix, can't repro, duplicate, stale Jul 25, 2022
@wilkinsona wilkinsona added status: declined A suggestion or change that we don't feel we should currently apply and removed status: pending-design-work Needs design work before any code can be developed labels Jul 25, 2022
@wilkinsona wilkinsona removed this from the 3.x milestone Jul 25, 2022
@rstoyanchev
Copy link
Contributor

Note that there is now spring-projects/spring-framework#29296, which will likely give us a better model for dealing with multiple HttpServiceProxyFactory instances for different remotes.

@bclozel
Copy link
Member

bclozel commented Oct 17, 2022

Reopening because of changes made in spring-projects/spring-framework#29296
Spring Boot could contribute a pre-configured HttpServiceProxyFactory.Builder to the context so developer can build their own client from it.

@bclozel bclozel reopened this Oct 17, 2022
@bclozel bclozel added status: waiting-for-triage An issue we've not yet triaged and removed status: declined A suggestion or change that we don't feel we should currently apply labels Oct 17, 2022
@bclozel bclozel added this to the 3.0.x milestone Oct 18, 2022
@bclozel bclozel removed the status: waiting-for-triage An issue we've not yet triaged label Oct 18, 2022
@bclozel bclozel modified the milestones: 3.0.x, 3.x Oct 18, 2022
@hannah23280
Copy link

hannah23280 commented Nov 13, 2022

With regard to this nice tutorial on HTTP Interface https://softice.dev/posts/introduction_to_spring_framework_6_http_interfaces/,

I don't quite understand. Why do developer need to manually write a @Bean method that will return the proxy bean (which implement the interface) especially if we are using spring boot?
I recall using @FeignClient, I do not have to define any proxy bean for it, so I presume spring boot will do for us.

Also why would one use Http Interface over @FeignClient?

@DanielLiu1123
Copy link
Contributor

DanielLiu1123 commented Jan 13, 2023

We could even think about an annotation with which the HTTP interface can be annotated and then directly injected into a consumer (like @FeignClient).

I think we need an annotation like @EnableFeignClients rather than @FeignClient.

We can already know whether an interface is a http client through @HttpExchange, we need an annotation to scan the interfaces and register beans (like @EnableFeignClients).

Here‘s my workaround.

@alsikorski
Copy link

I think people have gotten used to using the Feign client approach.

Here you can find very similar aproach: Exchange client

It is really simple to use.

@maciejwalkowiak
Copy link
Contributor

maciejwalkowiak commented Apr 4, 2023

I have prototyped following approach, that reduces the boilerplate to minimum:

@HttpClient annotation to mark interface as an http client and add option to set the WebClient bean name to use.

@HttpClient("todo-client")
public interface TodoClient {
    @GetExchange("/todos")
    List<Todo> get();
}

This annotation is processed by an registrar implementing ImportBeanDefinitionRegistrar that registers bean definition for each http client with HttpServiceProxyFactory and WebClientAdapter creating an adapter for a WebClient with a name from the annotation.

Creating WebClient instances from the environment

Considering that many web clients are relatively simple, there is a common set of properties that can be set with simple properties: url, basic auth, timeouts etc.

Given this, there is an option to create WebClients through yaml/properties like this:

http.clients:
    todo-client:
        url: https://jsonplaceholder.typicode.com
    bar:
        url: http://foo/bar

If you believe it makes sense I can prepare a PR or if it's too early to say I can release it as a separate project that will get deprecated once Spring Boot has similar functionality built in.

Update:

The library is available on Maven Central: https://github.com/maciejwalkowiak/spring-boot-http-clients

@OlgaMaciaszek
Copy link
Contributor

Another proposed implementation: https://github.com/joshlong/declarative-client-registration by @joshlong .

@DanielLiu1123
Copy link
Contributor

DanielLiu1123 commented Dec 13, 2023

Let me introduce once again to httpexchange-spring-boot-starter, which is probably the most comprehensive implementation I could find.

This project is entirely configuration-driven and can achieve the same functionalities as Spring Cloud OpenFeign without introducing any external annotations. Including setting different baseUrl/timeout for each client, integration with Spring Cloud LoadBalancer, dynamic refresh, and more.

http-exchange:
  base-packages: [com.example]
  connect-timeout: 1000
  read-timeout: 3000
  client-type: rest_client
  channels:
    - base-url: http://order
      clients:
        - com.example.order.api.*
    - base-url: http://user
      read-timeout: 5000
      clients:
        - com.example.user.api.*

The main goals of this project:

  • Promote the use of @HttpExchange as a neutral annotation to define API interfaces.
  • Provide a Spring Cloud OpenFeign like experience for Spring 6.x declarative HTTP clients.
  • Support @RequestMapping based annotations (easy to migrate from Spring Cloud OpenFeign).
  • Not introduce external annotations, easy to migrate to other implementations.

It's definitely worth a try!

@hannah23280
Copy link

Let me introduce once again to httpexchange-spring-boot-starter, which is probably the most comprehensive implementation I could find.

This project is entirely configuration-driven and can achieve the same functionalities as Spring Cloud OpenFeign without introducing any external annotations. Including setting different baseUrl/timeout for each client, integration with Spring Cloud LoadBalancer, dynamic refresh, and more.

http-exchange:
  base-packages: [com.example]
  connect-timeout: 1000
  read-timeout: 3000
  client-type: rest_client
  channels:
    - base-url: http://order
      clients:
        - com.example.order.api.*
    - base-url: http://user
      read-timeout: 5000
      clients:
        - com.example.user.api.*

The main goals of this project:

  • Promote the use of @HttpExchange as a neutral annotation to define API interfaces.
  • Provide a Spring Cloud OpenFeign like experience for Spring 6.x declarative HTTP clients.
  • Support @RequestMapping based annotations (easy to migrate from Spring Cloud OpenFeign).
  • Not introduce external annotations, easy to migrate to other implementations.

It's definitely worth a try!

Great Effort!! Wish the spring boot will include your starter as an official one!

@hannah23280
Copy link

So is there any plan or any work in progress to include this feature natively?

@OlgaMaciaszek
Copy link
Contributor

We're discussing it within the team now. We will update here once decision's been taken.

@FilipBehnke
Copy link

@OlgaMaciaszek are there any updates on this topic?

@OlgaMaciaszek
Copy link
Contributor

Internal POC planned for this month to discussed with the team. Will post any updates here.

@XhstormR
Copy link

XhstormR commented May 3, 2024

@OlgaMaciaszek Since spring-cloud-openfeign is going to enter maintenance mode, will this become the alternative options to openfeign, any update for now?

@ZaheerUdDeen
Copy link

Hey Folks, any updates here. when the declrative approach will be available natively?

@OlgaMaciaszek
Copy link
Contributor

Working on POC at this point. Once that's done we'll able to hold a discussion on adding it to a backlog of a specific release. Will post any relevant updates here.

@yuexueyang
Copy link

@OlgaMaciaszek Thank you for introduce me to this post. httpexchange-spring-boot-starter is really helpful, but may be it still have a small difference from what I want. That's we always configure the base url in database, not in property files, and we need to obtain the base url during every call to the HttpExchange interface to ensure that we use the proper value for each customer (Yes, different customer may use different base url, and the remote site is not constucted by ourself)

@xtyuns
Copy link

xtyuns commented May 24, 2024

不同的客户可能使用不同的基 URL

Maybe you can do it with an org.springframework.http.client.ClientHttpRequestInterceptor

@spencergibb
Copy link
Member

Always get your configuration from the spring environment and use different configuration sources to load from a property or DB or whatever

@yuexueyang
Copy link

不同的客户可能使用不同的基 URL

Maybe you can do it with an org.springframework.http.client.ClientHttpRequestInterceptor
In my experience, this interceptor can only modify header values, not url.

@yuexueyang
Copy link

Always get your configuration from the spring environment and use different configuration sources to load from a property or DB or whatever

Thank you for your advice, what you mean is to configurer Proxy instance per customer for specified HttpExchange at startup, like Bean instance A for customer A, Bean instance B for customer B, and so on. Is this understanding correct?

@xtyuns
Copy link

xtyuns commented May 24, 2024

In my experience, this interceptor can only modify header values, not url.

You can replace the request, just like the example with kotlin:

val urlPrefixedInterceptor = ClientHttpRequestInterceptor { request, body, execution ->
    execution.execute(object : HttpRequest by request {
        override fun getURI(): URI {
            return URI.create("https://example.org${request.uri}")
        }
    }, body)
}

@notzero1995
Copy link

notzero1995 commented Nov 4, 2024

In my experience, this interceptor can only modify header values, not url.

You can replace the request, just like the example with kotlin:

val urlPrefixedInterceptor = ClientHttpRequestInterceptor { request, body, execution ->
    execution.execute(object : HttpRequest by request {
        override fun getURI(): URI {
            return URI.create("https://example.org${request.uri}")
        }
    }, body)
}

Good idea; example of Java code

public class DemoUrlInterceptor implements ClientHttpRequestInterceptor {
    @Override
    public ClientHttpResponse intercept(HttpRequest request, byte[] body, ClientHttpRequestExecution execution) throws IOException {
        HttpRequest newRequest = new HttpRequestWrapper(request) {
            @Override
            public URI getURI() {
                return URI.create("https://example.org" + request.getURI());
            }
        };
        return execution.execute(newRequest, body);
    }
}

@ch4mpy
Copy link

ch4mpy commented Nov 21, 2024

I ran some experiments around @HttpExchange proxies too. The following is released on maven-central.

I took a pretty different direction than httpexchange-spring-boot-starter for two reasons:

  • where I had the most to gain was with the underlying RestClient or WebClient configuration (mostly requests authorization and HTTP proxy configuration, which can both be cumbersome).
  • @HttpExchange interfaces are something I usually generate from OpenAPI specs. Decorating generated code with configuration annotations is not quite an option, reason why I prefer YAML in this case.

I believe that it's a good thing to de-couple the client bean definition from the generated @HttpExchange proxy one. The generated proxy needs a client, not to know how it is configured. So if the generated proxy bean definition contains a client bean reference instead of inlined client configuration:

  • A generated service proxy can use a hand-crafted client (for instance when we want some configuration not provided by the starter we use).
  • Several service proxies can use the same client bean.

I end with YAML for each client and generate no more than pre-configured RestClient or WebClient beans (or builders if more custom configuration is needed). This is not as magic as the @FeignClient auto-detection and implementation, but is great step forward in usability for my use cases. Sample taken from this repo:

com:
  c4-soft:
    springaddons:
      rest:
        client:
          # Exposes a RestClient bean named machinClient (or WebClient in a WebFlux app)
          machin-client:
            base-url: ${machin-api}
            authorization:
              oauth2:
                # Authorize outgoing requests with the Bearer token in the security context (possible only in a resource server app)
                forward-bearer: true
          # Exposes a RestClient.Builder bean named biduleClientBuilder (mind the "expose-builder: true")
          bidule-client:
            base-url: ${bidule-api}
            # Expose the builder instead of an already built client (to fine tune its conf)
            expose-builder: true
            authorization:
              oauth2:
                # Authorize outgoing requests with the Bearer token obtained using an OAuth2 client registration
                oauth2-registration-id: bidule-registration

Once the REST clients are configured, producing the @HttpExchange proxies we are all interested in is a snap:

@Configuration
public class RestConfiguration {
  /** 
   * @param machinClient pre-configured by spring-addons-starter-rest using application properties
   * @return a generated implementation of the {@link MachinApi} {@link HttpExchange &#64;HttpExchange}, exposed as a bean named "machinApi".
   */
  @Bean
  MachinApi machinApi(RestClient machinClient) throws Exception {
    return new RestClientHttpExchangeProxyFactoryBean<>(MachinApi.class, machinClient).getObject();
  }

  /** 
   * @param biduleClientBuilder pre-configured using application properties
   * @return a {@link RestClient} bean named "biduleClient"
   */
  @Bean
  RestClient biduleClient(RestClient.Builder biduleClientBuilder) throws Exception {
    // Fine-tune biduleClientBuilder configuration here
    return biduleClientBuilder.build();
  }

  /** 
   * @param biduleClient the bean exposed just above
   * @return a generated implementation of the {@link BiduleApi} {@link HttpExchange &#64;HttpExchange}, exposed as a bean named "biduleApi".
   */
  @Bean
  BiduleApi biduleApi(RestClient biduleClient) throws Exception {
    return new RestClientHttpExchangeProxyFactoryBean<>(BiduleApi.class, biduleClient).getObject();
  }
}

This exposes two generated @HttpExchange beans named machinApi and biduleApi, each with a different OAuth2 authorization mechanism (one forwarding the access token in the security context and the other using a client registration with client credentials). And if the HTTP_PROXY and NO_PROXY environment variables are set, the auto-configured REST clients would use it!

The next step could be replacing most of the Java code above ( keep just the biduleClientBuilder tuning) with something like:

        service:
          machin-service:
            client: machinClient
            exchange-class: pf.c4soft.MachinApi
          bidule-service:
            client: biduleClient
            exchange-class: pf.c4soft.BiduleApi

But I couldn't find a way to post-process the bean definition registry to do so - and the gains would be marginal.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type: enhancement A general enhancement
Projects
None yet
Development

Successfully merging a pull request may close this issue.