TASSTA Documentation Center TASSTA Documentation Center More products
Hide table of contents Hide details Search My account

HTTP error 503 instead of T.Commander

Issue Description

In some situations, the T.Commander web UI becomes unavailable and HTTP error 503 is shown instead, as in the following screenshot:

Error 503

Cause

These symptoms mean that the T.Commander UI has failed to start, although the T.Commander daemon is running.

Solution

This issue is resolved in T.Commander 3.5.4.1-13. If you are using an earlier version, upgrade T.Commander to to version 3.5.4.1-13 or later to prevent this from happening in the future.

If you want to fix the issue before the upgrade, take the following steps:

  1. Connect through SSH to the IP address of the server that has given you error 503 instead of access to T.Commander.
  2. Restart the T.Commander Service Server using the following command with root privileges:
    restart-commander

Check the logs for errors that may have occurred during the server restart. For that, use:

log-commander

If there are no errors in the logs, T.Commander should become accessible at https://:4321. If you encounter errors, see the sections below for details about resolving them.

If you get a "Waiting for changelog lock" error

You may get an error like the following in the log:

Oct 13 07:15:55 yourserverAM[35614]: [main] INFO liquibase -Waiting for changelog lock....

It means that the T.Commander database is locked. This may be a temporary lock applied by another T.Commander instance during its startup to prevent database schema inconsistency. In a worse scenario, it is a permanent lock that has to be lifted manually. This can happen if a T.Commander instance shuts down in a non-graceful way.

Check if there are any running T.Commander instances in the cluster. Run one of the following commands on both servers:

status-commander

– or –

systemctl status tassta_commander.service

If the T.Commander service is running on the node, the result looks like this:

Service is running

If the T.Commander service is not running, it looks like this:

Service is not running

If the service is not running, start it. Use one of the following commands:

start-commander

– or –

systemctl start tassta_commander.service

If the service start fails, or if the service is running but the database is still locked, see Unlocking the database below.

Unlocking the database

Use SSH to log into the server that has the MASTER state. Run the following command:

mysql

This takes you to the MySQL command line. Type and run the following snippet:

use tcommander;
UPDATE DATABASECHANGELOGLOCK SET LOCKED=FALSE, LOCKGRANTED=null, LOCKEDBY=null where ID=1;
Caution:

This is a potentially dangerous operation if done incorrectly. If you decide to perform it, make sure you know exactly what you are doing. Consider backing up the database prior to the operation.

After this, exit the MySQL command line:

exit

As a result, the database will become unlocked. You can proceed to start the T.Commander service using one of the following commands:

start-commander

– or –

systemctl start tassta_commander.service

Confirm that the T.Commander status is active using one of the following commands:

status-commander

– or –

systemctl status tassta_commander.service