Telegram Connection Manager (Morse) released.

I am happy to announce the second release of Telepathy-Morse — Qt-based Telepathy Connection Manager for Telegram network.

If you don’t know about the project, then you can read the previous post with an introduction.

Please note that although I also work on KDE Telepathy, there is no any dependency on KDE, so the Connection Manager can benefit Empathy, Jolla (Messages), Ubuntu IM client and any other software, powered by Telepathy.

Since the last post, the project got a basic group chat implementation, migrated to native (telegram) identifiers, got two-step verification support and so on. If you want to try the CM right now, then skip a couple of paragraphs and scroll down to the Links section at the end of this post.

Let’s dive into details.

Features overview

What is expected to work:

  • Contact list with first/last names
  • Contact avatars
  • Personal chat (one to one)
  • Simple group chat (no creation and no channels, see details below)
  • User typing events
  • Full message delivery status support (details below)
  • Own presence (online, offline, hidden)
  • Two-step verification (details below)
  • Loading unread messages on connect
  • Sessions (Means that you don’t have to get confirmation code again and again)
  • Restoring connection on network problem

What does not work:

  • Contact management. You can remove a contact for contact list, but you can not add they. Telepathy requires a contact identifier to add it to your contact list; Telegram gives you the contact identifier only as a result of adding it to your contact list by a phone number. Phone number can not be used as an identifier in Telepathy-Morse, because the contacts phone number can be unknown, can be changed or become unavailable and so on. Looks like a circle or a deadlock. Thanks to Robert McQueen for the workaround idea: we can introduce a “tel:+12345..” identifiers, which will be suitable only for subscription requests. I will try to implement this in the next version.
  • Multimedia messages do not work, except Geo. See details below.

Details

Two-step verification

Two months ago I added two-step verification support to TelegramQt and Morse and found that this part of Telepathy can be significantly improved. Currently Telepathy spec offers two authentication interfaces: SASLAuthentication and CaptchaAuthentication. Captcha is obviously not what we need for Telegram, so we have to use SASLAuthentication and there is a limitation, which badly affects UX: it is not possible to distinguish authentication channels (requests), which means that there is no visual difference between phone code and password requests.

I would like to bring all Telegram features to Telepathy, which means that in long-term we need to support all methods to obtain the auth code. As far as I know, Telegram user can request an in-application code, a SMS message with code or a voice call to the account phone number (“a robot will repeat the confirmation code from a previously sent SMS message.”). For Telepathy it means that the set of available authentication methods can be changed dynamically (“SendCall” usually becomes available only after “SendSMS”). As some of you can recall, there was been a plan to use an awesome project KDE Connect, to (semi-)automatically pass the received auth code to the ConnectionManager. In order to support all those features in a generic manner, I have a plan to develop OTP Authentication Channel Interface.

Group chat

Group chat creation is not supported, because Telepathy needs to get the new “room” identifier and handle right on creation, but we need to wait for Telegram server reply to get the result of the operation (the result will contain telegram identifier if the request is succeeded). This does not work because TelepathyQt does not support delayed DBus replies and I have no solid solution here yet.

As far as I know, Telepathy does not support “group chat list” (as it supports contact list), so I had to workaround it: Morse exposes group chat list in room search result.

join_chat

I’m planning to add Telegram Channel support sometime soon; I need to update the TelegramQt to newer MProto layer, but that should not be a problem.

Incredible (full) support of message (delivery) status

For a year Morse support follow standard things:

  • delivery status “Accepted” on message delivered to the server
  • delivery status “Read” on message read by the recipient
  • send “MessageRead” on message acknowledged by Telepathy client

But also you can:

  • Send a message from a device and see it “sent” in Telepathy client
  • Receive a message on all clients at the same time
  • Read a message from a device and get it removed from the Telepathy pending messages list by Morse. Please, let me know if such feature is already implemented in any other telepathy connection manager. Also, please take a note, that I had to change TpQt API for this feature and the change will be in TelepathyQt-0.9.8, which is not released yet.

cuts

Geopoint messages

I added geo message support in form of text/plain RFC 5870 and application/geo+json mime type to Morse and implemented a “filter” for the text/plain version for the KDE Telepathy Text UI, so received geo messages will appear in OpenStreetMap iframe with the “geo:” text replaced by a link. Zoom argument also implemented in KTp, but there is no such thing in Telegram, so the default zoom value will be used instead.

ktp-geo

As far as I see, it is not included in KDE Applications 16.08, but I’m sure it will be available in the upcoming 16.12 release. I disabled the filter by default for the first release, so you have to enable it via Settings->Configure Chat Application->Plugins->Geopoint Preview.

Telegram identifiers

I changed Morse contact and group chat identifiers to expose native Telegram userId/chatId. UserId is the most suitable way to identify contacts, because it never changes, which is important for proper logging.

  • If you left a chat and then join back, the chat identifier will be the same.
  • If you removed a contact from the contact list, the identifier will be the same (phone number will be unavailable).
  • If you add a contact with a long communication history to your contact list (effectively say Telegram that you know the contact phone number), the identifier will be the same.
  • If you or your contact changed a phone number, the identifier… yes, it will be…

ContactInfo interface

After Telepathy identifiers changed to be Telegram native identifiers, there was been no way to see phone numbers. I implemented Connection.Interface.ContactInfo to expose three (basic) contacts vCard fields:

  • nickname (username in terms of telegram)
  • tel (the phone number)
  • name (first and last name individually)

It turns out that KDE Telepathy does not support the ContactInfo interface after the introduction of KPeople. It seems that TelepathyQt Factories do not support optional interfaces, so Telepathy backend for KPeople does not require ContactInfo feature to be operatable with ConnectionManager without the ContactInfo interface. It is possible to make Groups interface optional, because it is exposed as Connection feature and Connection object is available at the time for Factory, but ContactInfo is exposed as Contact feature and API don’t let us to know in advance if the ContactInfo is supported. I hope that required changes would make it into TelepathyQt-0.9.8 release.

The feature already works in Empathy as intended.

Debug interface

I implemented Debug interface in TelepathyQt and Morse, so now it is much easier to get debug messages from the connection manager. I changed KTp Debugger to make it possible to debug initialization and authorization steps of connection. Previously debugger waited for success connection before attach and that was been weird. I will share more details about made and upcoming KTp Debugger improvements in one of the next posts.

debugger

Short-term goals

ContactSearch interface

I will try to implement ContactSearch interface; it should be easy enough, but to make it really useful and convenient we need to have a list of recent dialogs (Telegram Dialogs).

Connection / setAvatar()

Morse does not support the setAvatar() method yet, because at the time of the avatar interface implementation there was no support for media files upload in TelegramQt. I implemented the file upload support, so this method finally can be added.

Long-term goals

Multimedia messages

Multimedia messages support is mostly implemented in TelegramQt, but it will be a very complex task to bring this support to Telepathy. This task can take a year or more with current development speed. We need to change everything (except maybe MissionControl) to make this works right.

telegramqt_multimedia

TelegramQt supports multimedia messages (photos, stickers and more), but it can not be exposed via Telepathy yet

In theory, it can be possible to implement image preview (thumbnail) with the current Telepathy specification, I will check if it is actually possible sometime soon. It is not supported in clients, so even this small step would take a lot of time.

I have some ideas and I will experiment with different approaches. Maybe I’ll be managed to make a Proof of Concept this winter.

Telegram Dialogs

The feature does not fit into Telepathy specification. I see two ways:

  • I can expose contacts from Dialogs in a contact list group, such as “Recent contacts”. The feature would be toggled via account parameter, such as “EnableRecentContacts”. It is an open question how to manage group chat.
  • Implement a new Telepathy interface for the Dialogs feature. Proper, but a very long way, which requires support from telepathy clients. The interface would be hard to standardize in Telepathy, because it is Telegram specific.

OTP Authentication Channel Interface

The key feature of the new interface would be the PasswordSent signal, but it would have a number of other, essential changes from the SASL interface too.

This is my plan on how the new interface will work:

  • User adds an account. The only required parameter is a phone number.
  • User sets presence to online.
  • Morse creates an OTP authentication channel (VerificationStatus: Not_Started, AvailableMethods: “sms”, PhoneNumber: the user phone number)
  • Telepathy auth application handles the channel. I have no strong decision on the UI, so the app will either:
    • calls UseMethod(“sms”) and informs the user, that verification code will be sent via sms (because it is the only available method)
    • or show a button box with sms and call actions with the last one disabled.
  • The auth handler connects via DBus to KDE Connect service signal about incoming sms. It will be even possible to find the right device (thanks to PhoneNumber interface property).
  • Morse:
    • Makes request for sms.
    • Changes VerificationStatus to In_Progress
    • On server respond, Morse emits PasswordSent signal with details:
      • PasswordRegexp is five digits after space
      • SenderRegexp is “Telegram” or “TeleVerify” (I would like Telegram server to tell us the exact sender phone/name.)
      • CallTimeout (received from the server. Something like 30 sec)
  • Telepathy auth application:
    • shows countdown if password details have “Expiration” value.
    • waits for KDE Connect incoming sms signal
    • on KDE Connect signal, ktp-auth-handler matches sender against SenderRegexp and then match the message against PasswordRegexp. If both checks succeed, then it either automatically call Respond with the password, or, at least, insert the code to the input line and waits for the user action.
  • Morse (optionally):
    • adds “call” method to the available method list.
  • Telepathy auth application:
    • shows buttons for all available methods (“Request an SMS”, “Request a call”)
  • Morse (if two-factor authentication is enabled (which a popular and highly recommended case)):
    • creates a usual SASLAuthentication channel for the password authentication…

There is also one important trick needed to make it works: Morse needs to know if the client is capable to handle OTP channel, because it should wait for the user decision, if the OTP interface is available (whether to send the code in-app or via sms) and it should automatically start authorization via sms otherwise, to let user to auth via plain SASL interface. This thing would require a lot of work in many Telepathy components, so it is especially good to know that ClientCapabilities interface is already available and we don’t have to invent it.

Known Issues

  • Initial low-level encryption sometimes generates bad random values, which leads to “connection doesn’t work” issue.
  • Can not send long messages (Missed TelegramQt gzip packing implementation; limit is about 400 characters; telegram protocol limitation is 4095 characters)

Both TelegramQt and Telepathy-Morse are Qt4 and Qt5 compatible.
Information about CMake build: by default CMake looks for the Qt5 build. Use USE_QT4 option (-DUSE_QT4=true) to get Qt4 build.

I think I will drop Qt4 support since version 0.2.0, if there would be no strong reasons to support it.

Account setup

It is said that there are issues with account setup. I think that problems related to configuration tools and there is nothing to fix in Morse. I would suggest to use a command-line tool as a fallback:
[kaffeine@kaffeine-pc ~]$ mc-tool add morse/telegram KaffeineTelegram string:account=1234567890
morse/telegram/_312345678901
[kaffeine@kaffeine-pc ~]$ mc-tool enable morse/telegram/_312345678901
[kaffeine@kaffeine-pc ~]$

Replace KaffeineTelegram by your account display name and 1234567890 by your phone number (without leading plus).

Authentication on Sailfish OS

As far as I know, authentication handler just does not appear and I have no device with  Sailfish OS to debug it. I tried to use an emulator, but it did not work for me and I have tons of other things to do, so I didn’t spend hours on it.

It is possible to copy authorization key from your PC to the device, the directory is ~/.cache/telepathy-morse/secrets.

In order to fix the issue, I need to get the debug information. I implemented Debug interface in Morse and work on making its possible to use KTp Debugger on Sailfish OS. Though this is not a high-priority task, the fix is on its way, stay tuned for updates.

Thanks

  • Thanks to George Kiagiadakis for work on TelepathyQt and for the better name for the OTP Authentication Interface. I started to work on this interface named Channel.Interface.PhoneVerification, but suggested OTP seems to fit better.
  • Special thanks to David Edmundson!
  • Thanks to Telepathy developers and everyone in #telepathy irc channel on freenode.
  • Special thanks to Robert McQueen!
  • Thanks to the KDE community for awesome Telepathy Sprint in 2014 and Google Summer of Code after that.
  • That said, thanks to Google.
  • Thanks to Giovanni Santini and others for testing, adding issues and pinging me with “any progress?” and similar questions. You made me to work more on this project.

Thank you for reading!

Stay tuned

Stay tuned for the next posts:

  • New life for Telepathy Debugger. I would like to share my works on KDE Telepathy Debugger, which is getting a lot of improvements, which open possibility for QML interface and a port to mobile platforms.
  • Announce of a new, long-awaited interface for Telepathy. Comes with implementation for Morse and some NotAnnouncedYet Connection Manager (for a widely used protocol, which already has ConnectionManager with a lot of other features, but not this one). I also added the interface support into KDE Telepathy. Maybe someone will implement it on the telepathy-glib / Empathy side.

Links


Advertisements
Tagged with: , , ,
Posted in Uncategorized
9 comments on “Telegram Connection Manager (Morse) released.
  1. This is great news! I’m glad to see that not everyone has given up on getting Telegram to work in Telepathy, great work! 🙂

    One thing that I hope will get high priority is supporting the integration of group chats into the contact list.
    Every Telegram user I know is using Telegram heavily (or even mostly) for group chats, and using the “join chatroom” UI from KTp to open each group manually is really killing the experience. It works on paper, but I’ve tried it, and it’s so cumbersome that it makes you just go back to a different client.
    Making it possible to integrate existing group chats would not only help for Telegram, but e.g. for Hangouts as well, so it would be a great addition to (KDE) Telepathy.

    • Kaffeine says:

      Hello Thomas,

      The problem is that it is not possible to get a list of active group chats on Telepathy layer, so it is not a just question of polish Daniels changes (https://git.reviewboard.kde.org/r/118237/), but rather a question of a new Telepathy specification, (which is a complex task).
      There is also one related feature: Dialogs — a list of recent communications. Typical (Telegram) use-case: you find some contact via search or in a group chat, and speak with them. If at the next time you want to continue the conversation, then you can find it in this Dialog list. Our issue is that user can not add arbitrary contact to contact list, because such operation requires the contact phone number. Instead, user wants to speak with “a person with whom I spoke yesterday”, which is not possible yet.
      IMO we need to three lists (to show in ktp-contact-list): ContactList, RoomList and DialigList. The last one intersects with the first two, so probably we need a toggle Contact+Room list Dialogs.
      Dialogs Feature is also highly demanded for social networks (though we don’t have such Connection Managers any more after Facebook “retirement”).

      • That does indeed sound like a lot of work to do, but I’m convinced that this will be necessary if we want to keep Telepathy relevant.

        And although ad-hoc dialogs with people not in your contact list are not possible within Hangouts (or maybe they are, I don’t know), showing recent conversations within the contact list is very useful there as well, as the web and Android clients for Hangouts do that as well.

        In general sorting by recent conversations (regardless of whether they are 1:1 or group chatd) seems to be where IM clients are going, because people don’t want to look through their contact list every time they want to talk to someone they talk to regularly.

        I hope you will find solutions to these things, because then there might be a chance for KTp to stay relevant 🙂

  2. Blizzz says:

    Um, what about secret chats?

    • Kaffeine says:

      Not implemented and no one asked yet.
      I think that most of the code is already in project, because Secret chat uses the same mechanism as the regular connection to the server, but even if this would be implemented in TelegramQt, we will have to add the support to telepathy clients, so they will be able to 1) request the secret chat instead of the regular one and 2) distinguish secret and “unsafe” text channels with the same contact.

  3. […] de mensajería instantánea más popular por estos lares, y es que por un lado tenemos la primera versión de Morse, el complemento que integrará Telegram en KDE Telepathy (ojo que está en desarrollo); y por el […]

  4. […] Telegram Connection Manager (Morse) released. […]

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: