Let's you are trying to do something in ArgoCD such as creating an app and "POST failed with status code 504" is being returned.

~]$ argocd app create hello-world --repo https://github.com/foo/bar.git --path files --dest-namespace my-project --dest-server https://api.dev.openshift.example.com:6443 --project my-project --grpc-web 
FATA[0030] rpc error: code = Unknown desc = POST https://openshift-gitops-server-openshift-gitops.apps.example.com/application.ApplicationService/Create failed with status code 504

Or trying to add a repo and 504 is being returned.

~]$ argocd repo add git@github.com:foo/bar.git --ssh-private-key-path ~/.ssh/argocd --name bar-repo --project my-project
FATA[0030] rpc error: code = Unknown desc = POST https://openshift-gitops-server-openshift-gitops.apps.openshift.example.com:443/repository.RepositoryService/ValidateAccess failed with status code 504

This should mean that the source (github.com in this example) is unable to reach the destination (https://openshift-gitops-server-openshift-gitops.apps.example.com in this example). In this example, ArgoCD is running on OpenShift and the base URL to reach ArgoCD is https://openshift-gitops-server-openshift-gitops.apps.dev.example.com).

Are you using the correct URL?

Before we get into a bunch of troubleshooting, let’s just make sure the URL being requested is correct. The kubectl (Kubernetes) or oc (OpenShift) get routes command can be used to ensure there is a route matching the destination URL. In this example, there is a route containing openshift-gitops-server-openshift-gitops.apps.example.com.

~]$ oc get routes --namespace openshift-gitops
NAME                      HOST/PORT                                                              PATH   SERVICES                  PORT    TERMINATION          WILDCARD
kam                       kam-openshift-gitops.apps.openshift.example.com                               kam                       8443    passthrough/None     None
openshift-gitops-server   openshift-gitops-server-openshift-gitops.apps.openshift.example.com           openshift-gitops-server   https   reencrypt/Redirect   None

You can describe the route and ensure:

• The route is exposed on a router - if not, you probably will need to label the route (FreeKB - OpenShift - Label a route using the oc label command)
• The route is forwarding requests onto the gitops server service
• The route has an endpoint

~]$ oc describe route openshift-gitops-server --namespace openshift-gitops
Name:                   openshift-gitops-server
Namespace:              openshift-gitops
Created:                10 days ago
Labels:                 app.kubernetes.io/managed-by=openshift-gitops
                        app.kubernetes.io/name=openshift-gitops-server
                        app.kubernetes.io/part-of=argocd
                        route-type=internal
Annotations:            openshift.io/host.generated=true
Requested Host:         openshift-gitops-server-openshift-gitops.apps.openshift.example.com
                           exposed on router default-router (host router-default-router.apps.openshift.example.com) 4 minutes ago
Path:                   <none>
TLS Termination:        reencrypt
Insecure Policy:        Redirect
Endpoint Port:          https

Service:        openshift-gitops-server
Weight:         100 (100%)
Endpoints:      10.131.0.37:8080

Are you logged into ArgoCD?

Before issuing an argocd command you probably first used the argocd login command to log into ArgoCD. You will want to ensure that the URL being used here is the route that has been exposed.

argocd login --username admin --password <password> openshift-gitops-server-openshift-gitops.apps.openshift.example.com

Network Policies

The kubectl (Kubernetes) or oc (OpenShift) get networkpolicies command can then be see if there are any ingress network policies denying incoming requests in the namespace ArgoCD is in. There is no need to use the kubectl (Kubernetes) or oc (OpenShift) get egressnetworkpolicies command since this issue is with ingress requests, not egress requests.

~]$ oc get networkpolicies --namespace openshift-gitops
No resources found in openshift-gitops namespace.

NetNamespaces

The kubectl (Kubernetes) or oc (OpenShift) get netnamespaces can be used to see if the namespace ArgoCD is in has a dedicated egress IP address.

~]$ oc get netnamespaces openshift-gitops
NAME               NETID     EGRESS IPS
openshift-gitops   9740194

If not, you may need to patch the namespace to have a dedicated egress IP address.

~]$ oc patch netnamespace openshift-gitops --type merge --patch '{ "egressIPs": [ "10.84.189.2" ] }'
netnamespace.network.openshift.io/openshift-gitops  patched