Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[HOLY WAR] Rustified Backend #98

Merged
merged 33 commits into from
Nov 2, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
33 commits
Select commit Hold shift + click to select a range
7f61933
feat: updated backend response for profile endpoint
harshkhandeparkar Oct 11, 2024
e5fc090
feat: added db fn to get paper by id
harshkhandeparkar Oct 11, 2024
b5e29fd
refactor: split routing into separate files
harshkhandeparkar Oct 11, 2024
cc2b493
feat: edit paper details endpoint
harshkhandeparkar Oct 11, 2024
030b42c
feat: use the edit endpoint (expect a few more commits fixing bugs)
harshkhandeparkar Oct 11, 2024
d922181
fix: change get to post (this is not the commit i expected)
harshkhandeparkar Oct 11, 2024
481b6cd
featfix: did some stuff this is too late and i am exhausted
harshkhandeparkar Oct 11, 2024
f95efba
feat: first commit the previous transaction then get the new db entry
harshkhandeparkar Oct 12, 2024
fff7693
feat<partial>: upload endpoint
harshkhandeparkar Oct 12, 2024
1e62a47
feat: added the delete endpoint
harshkhandeparkar Oct 12, 2024
c970c35
fix: only delete uploaded papers
harshkhandeparkar Oct 12, 2024
b083437
feat: similar papers endpoint
harshkhandeparkar Oct 12, 2024
30a2033
feat: fully working `/similar` GET endpoint
harshkhandeparkar Oct 12, 2024
7d2c600
feat: show currently running OCR in the sidebar
harshkhandeparkar Oct 12, 2024
e882f8f
refactor: moved all path related stuff to pathutils module
harshkhandeparkar Oct 16, 2024
ef3ef29
feat: (hopefully) fully functional upload endpoint
harshkhandeparkar Oct 16, 2024
0f03898
feat: working(?) /edit handler
harshkhandeparkar Oct 16, 2024
b5adab3
docs: added some documentation to the functions
harshkhandeparkar Oct 16, 2024
bdf86d0
feat: cargo fmt
harshkhandeparkar Oct 16, 2024
f9e37c2
fix: fixed the dum dum, also simplified the code a lot
harshkhandeparkar Oct 17, 2024
3f99c5a
fix: allow unused in some places
harshkhandeparkar Oct 17, 2024
9bc5591
feat: ensure existence of system paths
harshkhandeparkar Oct 17, 2024
713896d
feat: edit details
harshkhandeparkar Oct 17, 2024
671b1b5
fix: changed course -> query
harshkhandeparkar Oct 17, 2024
1a1d0aa
feat: ensure creation of paths in a better way
harshkhandeparkar Oct 17, 2024
fde4476
fix: parse string instead
harshkhandeparkar Oct 17, 2024
c7084c5
feat: allow editing (but not deleting) of approved papers from admin …
harshkhandeparkar Oct 17, 2024
2241760
fix: fixed the unknown handling in exam and semester, fixed search
harshkhandeparkar Oct 17, 2024
1c7b73b
feat: move to rustls from native-tls
harshkhandeparkar Oct 24, 2024
0a241ce
feat: working Dockerfile with portable binary build
harshkhandeparkar Oct 24, 2024
b061d97
feat: moved the new backend into the backend folder
harshkhandeparkar Oct 24, 2024
566f1ab
feat: changed backend port to 8080 in metaploy config
harshkhandeparkar Oct 24, 2024
729a3e6
docs: updated readme
harshkhandeparkar Oct 24, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
40 changes: 29 additions & 11 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -60,7 +60,7 @@ IQPS was originally created by [Shubham Mishra](https://github.com/grapheo12) in
- Make the env file by copying the template: `cp .env.template .env`
- Fill the env variable and set `DB_HOST=localhost` for running locally for development
- Start the DB by running `docker compose -f docker-compose.dev.yaml up -d`
- Start the Go backend by running `go run .`
- Start the Rust backend by running `cargo run .`
3. Set up the frontend by running `pnpm install` and then `pnpm start` in the `frontend/` directory.
4. Profit.

Expand All @@ -77,20 +77,38 @@ IQPS was originally created by [Shubham Mishra](https://github.com/grapheo12) in
5. Optionally set up a Systemd service to start the wiki on startup or use this [deployment github workflow](./.github/workflows/deploy.yaml).

### Environment Variables
Environment variables can be set using a `.env` file. Use the `.env.template` files for reference.
Environment variables can be set using a `.env` file. Use the `.env.template` files for reference. See `backend/src/env.rs` for more documentation and types.

#### Backend
- `DB_PATH`: Path to the database file to use.
- `STATIC_FILES_URL`: The base URL for the static files (PDFs).
- `QPS_PATH`: The local path on the server to store question paper PDFs.
- `DB_NAME`: The name for your database.
- `DB_HOST`: `localhost` if using local postgres server or set your **own** database host address.
- `DB_PORT`: `5432` if using local postgres server, or set to your **own** database port.
- `DB_USER`: The access username for database.
- `DB_PASSWORD`: The access password corressponding to the user.
##### Database (Postgres)
- `DB_NAME`: Database name
- `DB_HOST`: Database hostname (eg: `localhost`)
- `DB_PORT`: Database port
- `DB_USER`: Database username
- `DB_PASSWORD`: Database password

##### Authentication
- `GH_CLIENT_ID`: Client ID of the Github OAuth app.
- `GH_CLIENT_SECRET`: Client secret of the Github OAuth app.
- `GH_ORG_NAME`: The name of the Github organization of the admins.
- `GH_ORG_TEAM_SLUG`: The URL slug of the Github org team of the admins.
- `JWT_SECRET`: A secret key/password for JWT signing. It should be a long, random, unguessable string.

##### Configuration
- `MAX_UPLOAD_LIMIT`: Maximum number of files that can be uploaded at once.
- `LOG_LOCATION`: The path to a local logfile.
- `STATIC_FILES_URL`: The URL of the static files server. (eg: `https://static.metakgp.org`)
- `STATIC_FILE_STORAGE_LOCATION`: The path to the local directory from which the static files are served.
- `UPLOADED_QPS_PATH`: A path relative to `STATIC_FILE_STORAGE_LOCATION` where the uploaded question papers will be stored. (eg: `iqps/uploaded`)
- `LIBRARY_QPS_PATH`: A path relative to `STATIC_FILE_STORAGE_LOCATION` where the library question papers are scraped and stored. (eg: `peqp/qp`)
- `SERVER_PORT`: The port on which the server listens.
- `CORS_ALLOWED_ORIGINS`: A comma (,) separated list of origins to be allowed in CORS.

#### Frontend
- `VITE_BACKEND_URL`: The IQPS backend URL. Use `http://localhost:5000` in development.
- `VITE_BACKEND_URL`: The IQPS backend URL. Use `http://localhost:8080` in development.
- `VITE_MAX_UPLOAD_LIMIT` The maximum number of files that can be uploaded at once. (Note: This is only a client-side limit)
- `VITE_GH_OAUTH_CLIENT_ID` The Client ID of the Github OAuth app.


## Contact

Expand Down
16 changes: 16 additions & 0 deletions backend-go-old/.env.template
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
STATIC_FILES_URL=https://static.metakgp.org
DB_NAME=
DB_HOST=
DB_PORT=
DB_USER=
DB_PASSWORD=
STATIC_FILES_STORAGE_LOCATION=/srv/static
UPLOADED_QPS_PATH=iqps/uploaded # Relative to `STATIC_FILES_STORAGE_LOCATION`. Final upload location will be /srv/static/iqps/uploaded
MAX_UPLOAD_LIMIT=10
GH_CLIENT_ID= # public token of the oauth app
GH_PRIVATE_ID= # Private token of the oauth app
JWT_SECRET= # JWT encryption secret
GH_ORG_NAME= # name of the org
GH_ORG_TEAM_SLUG= #URL friendly team Name
GH_ORG_ADMIN_TOKEN= #GH TOKEN OF AN ORG ADMIN
IQPS_LOG_LOCATION=./log/application.log
27 changes: 27 additions & 0 deletions backend-go-old/Dockerfile
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
FROM golang:1.22.4 AS builder

WORKDIR /src

COPY go.mod go.sum ./

RUN go mod download

COPY . .

RUN CGO_ENABLED=1 GOOS=linux go build -o ./build --tags "fts5" -a -ldflags '-linkmode external -extldflags "-static"' .

FROM alpine:latest AS app

RUN apk --no-cache add tzdata ca-certificates bash

ENV TZ="Asia/Kolkata"

WORKDIR /app

COPY metaploy/ ./

RUN chmod +x ./postinstall.sh

COPY --from=builder /src/build .

CMD ["./postinstall.sh", "./build"]
File renamed without changes.
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ services:
- PGPORT=${DB_PORT}
- PGHOST=${DB_HOST}
healthcheck:
test: ["CMD", "pg_isready", "-U", "${POSTGRES_USER}", "${POSTGRES_DB}"]
test: ["CMD", "pg_isready", "-U", "postgres"]
interval: 5s
timeout: 3s
retries: 20
Expand Down
34 changes: 34 additions & 0 deletions backend-go-old/docker-compose.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
services:
iqps-backend:
image: metakgporg/iqps-backend
container_name: iqps-backend
build: .
restart: always
env_file:
- .env
networks:
metaploy-network:
aliases:
- iqps-backend
metaploy-private-network:
volumes:
- ./logs:/var/log/iqps/logs
- ./db:/db
- nginx-config-volume:/etc/nginx/sites-enabled
- odins-vault:/srv/static

networks:
metaploy-network:
external: true
name: metaploy-network
metaploy-private-network:
external: true
name: metaploy-private-network

volumes:
nginx-config-volume:
external: true
name: metaploy-nginx-config-volume
odins-vault:
external: true
name: odins-vault
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
16 changes: 16 additions & 0 deletions backend-go-old/metaploy/iqps.metaploy.conf
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
upstream iqps_server {
server iqps-backend:5000;
}

server {
server_name iqps-server.metakgp.org;

location / {
proxy_pass http://iqps_server;
}

location /upload {
proxy_pass http://iqps_server;
client_max_body_size 50m;
}
}
14 changes: 14 additions & 0 deletions backend-go-old/metaploy/postinstall.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#!/bin/sh

cleanup() {
echo "Container stopped. Removing nginx configuration."
rm /etc/nginx/sites-enabled/iqps.metaploy.conf
}

trap 'cleanup' SIGQUIT SIGTERM SIGHUP

"${@}" &

cp ./iqps.metaploy.conf /etc/nginx/sites-enabled

wait $!
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
24 changes: 0 additions & 24 deletions backend-rs/.env.template

This file was deleted.

Loading