Feat: simplify docker compose

Feat: use docker healthcheck

.env.docker.example

@@ -5,6 +5,8 @@ VERSION=1
 # See "ports" in docker-compose.yml.
 SERVER_ADDRESS=0.0.0.0
 SERVER_PORT=9503
+# If you use nginx, then provide here name of client header. for example x-real-ip or x-forwarded-for
+REAL_IP_HEADER=
 
 MEMORY_LIMIT=256M
 TIMEZONE=UTC
@@ -66,10 +68,3 @@ DB_SERIALIZER=serialize
 DB_ENABLE_MIN_DATABASE=0
 # Enable file metadata cache
 DB_ENABLE_FILE_REFERENCE_DATABASE=0
-
-# HEALTHCHECK
-# If server stops responding to requests it will be stoped
-# Requests made each 30 seconds by default.
-HEALTHCHECK_ENABLED=0
-HEALTHCHECK_INTERVAL=30
-HEALTHCHECK_REQUEST_TIMEOUT=10

.env.example

@@ -6,6 +6,8 @@ VERSION=1
 # To recieve requests from the Internet change to 0.0.0.0 and add rule to your firewall (THIS WILL MAKE API UNSECURE!)
 SERVER_ADDRESS=127.0.0.1
 SERVER_PORT=9503
+# If you use nginx, then provide here name of client header. for example x-real-ip or x-forwarded-for
+REAL_IP_HEADER=
 
 MEMORY_LIMIT=256M
 TIMEZONE=UTC
@@ -64,10 +66,3 @@ DB_SERIALIZER=serialize
 DB_ENABLE_MIN_DATABASE=0
 # Enable file metadata cache
 DB_ENABLE_FILE_REFERENCE_DATABASE=0
-
-# HEALTHCHECK
-# If server stops responding to requests it will be stoped
-# Requests made each 30 seconds by default.
-HEALTHCHECK_ENABLED=0
-HEALTHCHECK_INTERVAL=30
-HEALTHCHECK_REQUEST_TIMEOUT=10

README.md

@@ -116,10 +116,7 @@ It's recommended to use http_build_query, when using GET requests.
 Telegram is event driven platform. For example:  every time your account receives a message you immediately get an update.
 There are multiple ways of [getting updates](https://docs.madelineproto.xyz/docs/UPDATES.html) in TelegramApiServer / MadelineProto:  
     1. [Websocket](#eventhandler-updates-webhooks)
-    2. Long Polling:   
+    2. Webhook: 
-        send request to getUpdates endpoint  
-        `curl "127.0.0.1:9503/api/getUpdates?data[limit]=3&data[offset]=0&data[timeout]=10.0" -g`  
-    3. Webhook: 
         Redirect all updates to your endpoint, just like bot api!  
         `curl "127.0.0.1:9503/api/setWebhook?url=http%3A%2F%2Fexample.com%2Fsome_webhook" -g `  
         Example uses urlencoded url in query.
@@ -199,9 +196,35 @@ curl --location --request POST '127.0.0.1:9503/api/downloadToResponse' \
 Also see: https://docs.madelineproto.xyz/docs/FILES.html#downloading-files
 
 ### Multiple sessions support
-**WARNING: running multiple sessions in one instance is unstable.**
+Its recommended to run every session in separate container. 
+
+To add more containers create `docker-compose.override.yml` file.
+Docker will [automatically merge](https://docs.docker.com/compose/multiple-compose-files/merge/) it with default docker-compose file.
+
+Example of `docker-compose.override.yml` with two additional containers/sessions (3 in total):
+```yaml
+services:
+    api-2:
+        extends:
+            file: docker-compose.base.yml
+            service: base-api
+        ports:
+            - "127.0.0.1:9512:9503"
+        command:
+            - "-s=session-2"
+    api-3:
+        extends:
+            file: docker-compose.base.yml
+            service: base-api
+        ports:
+            - "127.0.0.1:9513:9503"
+        command:
+            - "-s=session-3"
+
+```
+### Multiple sessions in one container (deprecated)
+**WARNING: running multiple sessions in one instance/container is unstable.**
 Crash/error in one session will crash all of them.
-Correct way: override docker-compose.yml and add containers with different ports and session names for each session.
 
 When running  multiple sessions, need to define which session to use for request.
 Each session stored in `sessions/{$session}.madeline`. Nested folders supported.
@@ -259,10 +282,10 @@ Each session stored in `sessions/{$session}.madeline`. Nested folders supported.
 **Examples:**
 * Session list: `http://127.0.0.1:9503/system/getSessionList`
 * Adding session: `http://127.0.0.1:9503/system/addSession?session=users/xtrime`
-* Removing session (session file will remain): `http://127.0.0.1:9503/system/removeSession?session=users/xtrime`
+* ~~Removing session (session file will remain): `http://127.0.0.1:9503/system/removeSession?session=users/xtrime`
-  Due to madelineProto issue its instance still might be in memory and continue working even after the remove.
+  Due to madelineProto issue its instance still might be in memory and continue working even after the remove.~~
-* Remove session file: `http://127.0.0.1:9503/system/unlinkSessionFile?session=users/xtrime`
+* ~~Remove session file: `http://127.0.0.1:9503/system/unlinkSessionFile?session=users/xtrime`
-    Don`t forget to logout and call removeSession first!
+    Don`t forget to logout and call removeSession first!~~
 * Close TelegramApiServer (end process): `http://127.0.0.1:9503/system/exit`
 
 Full list of system methods available in [SystemApiExtensions class](https://github.com/xtrime-ru/TelegramApiServer/blob/master/src/MadelineProtoExtensions/SystemApiExtensions.php)

config.php

@@ -7,6 +7,7 @@ $settings = [
     'server' => [
         'address' => (string)getenv('SERVER_ADDRESS'),
         'port' => (int)getenv('SERVER_PORT'),
+        'real_ip_header' => (string)(getenv('REAL_IP_HEADER') ?? ''),
     ],
     'telegram' => [
         'app_info' => [ // obtained in https://my.telegram.org
@@ -67,11 +68,6 @@ $settings = [
         'passwords' => (array)json_decode((string)getenv('PASSWORDS'), true),
         'bulk_interval' => (float)getenv('REQUESTS_BULK_INTERVAL')
     ],
-    'health_check' => [
-        'enabled' => (bool)filter_var((string)getenv('HEALTHCHECK_ENABLED'), FILTER_VALIDATE_BOOL),
-        'interval' => ((int)getenv('HEALTHCHECK_INTERVAL') ?: 30),
-        'timeout' => ((int)getenv('HEALTHCHECK_REQUEST_TIMEOUT') ?: 60),
-    ]
 ];
 
 if (empty($settings['telegram']['connection']['proxies']['\danog\MadelineProto\Stream\Proxy\SocksProxy'][0]['address'])) {

docker-compose.base.yml

@@ -0,0 +1,37 @@
+services:
+  base-api:
+    image: xtrime/telegram-api-server:latest
+    build:
+      context: .
+      dockerfile: Dockerfile
+    restart: unless-stopped
+    volumes:
+      - ./:/app-host-link
+    working_dir: /app-host-link
+    depends_on:
+      - mysql
+    environment:
+      WAIT_HOSTS: mysql:3306
+    logging:
+      driver: "json-file"
+      options:
+        max-size: "1024k"
+        max-file: "2"
+    healthcheck:
+      test: curl -f http://localhost:9503/system/healthcheck || exit 1
+      interval: 60s
+      timeout: 10s
+      retries: 2
+      start_period: 60s
+  base-mysql:
+    image: mariadb:11.1
+    restart: unless-stopped
+    volumes:
+      - ./.mysql:/var/lib/mysql
+    environment:
+      MARIADB_ALLOW_EMPTY_ROOT_PASSWORD: 'yes'
+      MARIADB_AUTO_UPGRADE: 'yes'
+    command:
+      - --skip-grant-tables
+      - --innodb-buffer-pool-size=128M
+      - --wait_timeout=65

docker-compose.dev.yml

@@ -1,5 +1,8 @@
 services:
   api:
+    extends:
+      file: docker-compose.base.yml
+      service: base-api
     image: xtrime/telegram-api-server:dev
     build:
       context: .
@@ -9,3 +12,12 @@ services:
       - "9003"
     environment:
       PHP_IDE_CONFIG: "serverName=Docker"
+  mysql:
+    extends:
+      file: docker-compose.base.yml
+      service: base-mysql
+    ports:
+      - "127.0.0.1:9507:3306"
+networks:
+  default:
+    name: telegram-api-server

docker-compose.yml

@@ -1,41 +1,18 @@
 services:
   api:
-    image: xtrime/telegram-api-server:latest
+    extends:
-    build:
+      file: docker-compose.base.yml
-      context: .
+      service: base-api
-      dockerfile: Dockerfile
-    init: true
-    restart: unless-stopped
     ports:
       - "127.0.0.1:9503:9503"
-    volumes:
-      - ./:/app-host-link
-    working_dir: /app-host-link
-    depends_on:
-      - mysql
-    environment:
-      WAIT_HOSTS: mysql:3306
-    logging:
-      driver: "json-file"
-      options:
-        max-size: "1024k"
-        max-file: "2"
     command:
       - "-s=session"
   mysql:
-    image: mariadb:11.1
+    extends:
-    restart: unless-stopped
+      file: docker-compose.base.yml
+      service: base-mysql
     ports:
       - "127.0.0.1:9507:3306"
-    volumes:
-      - ./.mysql:/var/lib/mysql
-    environment:
-      MARIADB_ALLOW_EMPTY_ROOT_PASSWORD: 'yes'
-      MARIADB_AUTO_UPGRADE: 'yes'
-    command:
-     - --skip-grant-tables
-     - --innodb-buffer-pool-size=128M
-     - --wait_timeout=65
 networks:
   default:
     name: telegram-api-server

server.php

@@ -1,12 +1,7 @@
 <?php
 
-use TelegramApiServer\Config;
 use TelegramApiServer\Files;
-use TelegramApiServer\Migrations\SessionsMigration;
 use TelegramApiServer\Migrations\StartUpFixes;
-use TelegramApiServer\Migrations\SwooleToAmpMigration;
-use TelegramApiServer\Server\Fork;
-use TelegramApiServer\Server\HealthCheck;
 
 if (PHP_SAPI !== 'cli') {
     throw new RuntimeException('Start in CLI');
@@ -68,18 +63,6 @@ Example:
 
 require_once __DIR__ . '/bootstrap.php';
 
-
-$mainProcessPid = getmypid();
-
-if (Config::getInstance()->get('health_check.enabled')) {
-    if (!defined('SIGINT')) {
-        throw new RuntimeException('pcintl required for healthcheck. Use docker.');
-    }
-    Fork::run(static function () use ($mainProcessPid) {
-        HealthCheck::start($mainProcessPid);
-    });
-}
-
 $sessions = [];
 foreach ($options['session'] as $session) {
     $session = trim($session);

src/MadelineProtoExtensions/SystemApiExtensions.php

@@ -22,6 +22,22 @@ class SystemApiExtensions
         $this->client = $client;
     }
 
+    /**
+     * @return array<string, array>
+     */
+    public function healthcheck(): array
+    {
+        $results = [];
+        ['sessions' => $sessions] = $this->getSessionList();
+        foreach ($sessions as $sessionKey => $session) {
+            $instance = $this->client->instances[$sessionKey];
+            if ($instance->getAuthorization() === API::LOGGED_IN) {
+                $results[$sessionKey] = $instance->fullGetSelf();
+            }
+        }
+        return $results;
+    }
+
     public function addSession(string $session, array $settings = []): array
     {
         if (!empty($settings['app_info']['api_id'])) {

src/Server/Authorization.php

@@ -30,7 +30,11 @@ class Authorization implements Middleware
 
     public function handleRequest(Request $request, RequestHandler $requestHandler): Response
     {
-        [$host] = explode(':', $request->getClient()->getRemoteAddress()->toString(), 2);
+        $host = Server::getClientIp($request);
+
+        if ($this->isLocal($host)) {
+            return $requestHandler->handleRequest($request);
+        }
 
         if ($this->passwords) {
             $header = (string)$request->getHeader('Authorization');
@@ -54,6 +58,19 @@ class Authorization implements Middleware
 
     private function isIpAllowed(string $host): bool
     {
+
+
+        if ($this->ipWhitelist && !in_array($host, $this->ipWhitelist, true)) {
+            return false;
+        }
+        return true;
+    }
+
+    private function isLocal(string $host): bool {
+        if ($host === '127.0.0.1' || $host === 'localhost') {
+            return true;
+        }
+
         global $options;
         if ($options['docker']) {
             $isSameNetwork = abs(ip2long($host) - $this->selfIp) < 256;
@@ -61,10 +78,6 @@ class Authorization implements Middleware
                 return true;
             }
         }
-
-        if ($this->ipWhitelist && !in_array($host, $this->ipWhitelist, true)) {
         return false;
     }
-        return true;
-    }
 }

src/Server/Fork.php

@@ -1,21 +0,0 @@
-<?php
-
-namespace TelegramApiServer\Server;
-
-use RuntimeException;
-
-class Fork
-{
-    public static function run(callable $callback)
-    {
-        $pid = pcntl_fork();
-        if ($pid === -1) {
-            throw new RuntimeException('Could not fork');
-        }
-        if ($pid !== 0) {
-            return;
-        }
-        $callback();
-        exit;
-    }
-}

src/Server/HealthCheck.php

@@ -1,134 +0,0 @@
-<?php
-
-namespace TelegramApiServer\Server;
-
-use Amp\Future;
-use Amp\Http\Client\HttpClientBuilder;
-use Amp\Http\Client\Request;
-use Revolt\EventLoop;
-use RuntimeException;
-use TelegramApiServer\Config;
-use TelegramApiServer\Logger;
-use Throwable;
-use UnexpectedValueException;
-use function Amp\async;
-use function Amp\Future\awaitAll;
-use function Amp\trapSignal;
-
-class HealthCheck
-{
-    private static string $host = '127.0.0.1';
-    private static int $port = 9503;
-    private static int $checkInterval = 30;
-    private static int $requestTimeout = 60;
-
-    /**
-     * Sends requests to /system and /api
-     * In case of failure will shut down main process.
-     *
-     * @param int $parentPid
-     *      Pid of process to shut down in case of failure.
-     */
-    public static function start(int $parentPid): void
-    {
-        static::$host = (string)Config::getInstance()->get('server.address');
-        if (static::$host === '0.0.0.0') {
-            static::$host = '127.0.0.1';
-        }
-        static::$port = (int)Config::getInstance()->get('server.port');
-
-        static::$checkInterval = (int)Config::getInstance()->get('health_check.interval');
-        static::$requestTimeout = (int)Config::getInstance()->get('health_check.timeout');
-
-        EventLoop::repeat(static::$checkInterval, static function () use ($parentPid) {
-            try {
-                Logger::getInstance()->info('Start health check');
-                if (!self::isProcessAlive($parentPid)) {
-                    throw new RuntimeException('Parent process died');
-                }
-                $sessions = static::getSessionList();
-                $sessionsForCheck = static::getLoggedInSessions($sessions);
-                $futures = [];
-                foreach ($sessionsForCheck as $session) {
-                    $futures[] = static::checkSession($session);
-                }
-                awaitAll($futures);
-
-                Logger::getInstance()->info('Health check ok. Sessions checked: ' . count($sessionsForCheck));
-            } catch (Throwable $e) {
-                Logger::getInstance()->error($e->getMessage());
-                Logger::getInstance()->critical('Health check failed');
-                if (self::isProcessAlive($parentPid)) {
-                    Logger::getInstance()->critical('Killing parent process');
-
-                    exec("kill -2 $parentPid");
-                    if (self::isProcessAlive($parentPid)) {
-                        exec("kill -9 $parentPid");
-                    }
-                }
-                exit(1);
-            }
-        });
-
-        trapSignal([SIGINT, SIGTERM]);
-        Logger::getInstance()->critical('Health check process exit');
-
-    }
-
-    private static function getSessionList(): array
-    {
-        $url = sprintf("http://%s:%s/system/getSessionList", static::$host, static::$port);
-
-        $response = static::sendRequest($url);
-
-        if ($response === false) {
-            throw new UnexpectedValueException('No response from /system');
-        }
-
-        return json_decode($response, true, 10, JSON_THROW_ON_ERROR)['response']['sessions'];
-    }
-
-    private static function getLoggedInSessions(array $sessions): array
-    {
-        $loggedInSessions = [];
-        foreach ($sessions as $sessionName => $session) {
-            if ($session['status'] === 'LOGGED_IN') {
-                $loggedInSessions[] = $sessionName;
-            }
-        }
-
-        return $loggedInSessions;
-    }
-
-    private static function checkSession(string $sessionName): Future
-    {
-        return async(function () use ($sessionName) {
-            $url = sprintf("http://%s:%s/api/%s/getSelf", static::$host, static::$port, $sessionName);
-            $response = static::sendRequest($url);
-            $response = json_decode($response, true, 10, JSON_THROW_ON_ERROR);
-            if (empty($response['response'])) {
-                Logger::getInstance()->error('Health check response: ', $response);
-                throw new RuntimeException("Failed health check: $url");
-            }
-            return $response;
-        });
-    }
-
-
-    private static function sendRequest(string $url): string
-    {
-        $client = (new HttpClientBuilder)::buildDefault();
-        $request = new Request($url);
-        $request->setInactivityTimeout(static::$requestTimeout);
-        $request->setTransferTimeout(static::$requestTimeout);
-
-        $response = $client->request($request);
-        return $response->getBody()->buffer();
-    }
-
-    private static function isProcessAlive(int $pid): bool
-    {
-        $result = exec("ps -p $pid | grep $pid");
-        return !empty($result);
-    }
-}

src/Server/Server.php

@@ -92,6 +92,16 @@ class Server
 
     public static function getClientIp(Request $request): string
     {
+        $realIpHeader = Config::getInstance()->get('server.real_ip_header');
+        if ($realIpHeader) {
+            $remote = $request->getHeader($realIpHeader);
+            if (!$remote) {
+                GOTO DIRECT;
+            }
+            $tmp = explode(',', $remote);
+            $remote = trim(end($tmp));
+        } else {
+            DIRECT:
             $remote = $request->getClient()->getRemoteAddress()->toString();
             $hostArray = explode(':', $remote);
             if (count($hostArray) >= 2) {
@@ -101,6 +111,8 @@ class Server
                 }
             }
 
+        }
+
         return $remote;
     }