Jump to content
amblingbear

Failure on bulding FriendCore on Manjaro plus install.sh feedback

Recommended Posts

Firstly, just to say that FriendOS is such a neat idea!  I hope this feedback is useful.

-- TLDR:

FriendCore linking fails on Manjaro - any help appreciated:

++++++++++++++++
cc: error: ../libs-ext/openssl/libssl.a: No such file or directory
gcc: error: ../libs-ext/openssl/libcrypto.a: No such file or directory
make[1]: *** [Makefile:201: bin/FriendCore] Error 1
make[1]: Leaving directory '/home/friend/friendup/core'
make: *** [Makefile:157: release] Error 2

++++++++++++++++

 

-- The longer read with feedback and workarounds:

Please note: I am an app support analyst and support engineer, and not a hardcore developer, so I hope that in some way this feedback is useful. Ultimately I was unable to build FriendCore on Manjaro.

I did get to see the workspace after editing the install on a previous attempt on Ubuntu 18.04 (though this was far from a clean install experience), but it only ever started up and launched the workspace _once_. I thought to try on Arch linux too to see if this was any cleaner and more reliable (both in building the FriendCore server and launching the webclient against it). I wasn't as successful on Arch though I have extracts of my experience below and hope this may help you in some way.


My system:
Linux xen 5.4.22-1-MANJARO #1 SMP PREEMPT Mon Feb 24 09:01:11 UTC 2020 x86_64 GNU/Linux

NAME="Manjaro Linux"
ID=manjaro
ID_LIKE=arch
PRETTY_NAME="Manjaro Linux"
ANSI_COLOR="1;32"
HOME_URL="https://www.manjaro.org/"
SUPPORT_URL="https://www.manjaro.org/"
BUG_REPORT_URL="https://bugs.manjaro.org/"
LOGO=manjarolinux


On running https://github.com/FriendUPCloud/friendup cloned install.sh:

# install patch and other devel tooling
pacman -S base-devel dialog    ## needed later for fakeroot binary but also provides the patch binary - suggest for inclusion in the install script

- unexpected output in Arch install  
sudo: apt-get: command not found


- Error

++++++++++++++++
libmatheval.tar.gz                [ <=>                                              ]   1.69K  --.-KB/s    in 0s      

2020-03-02 20:43:31 (10.6 MB/s) - ‘libmatheval.tar.gz’ saved [1730]

libmatheval/
libmatheval/.SRCINFO
libmatheval/PKGBUILD
libmatheval/removeifndefs.patch
libmatheval/replace-obsolete-functions.patch
patch: **** Can't open patch file ../patches/archlinux-libmatheval-PKGBUILD.patch : No such file or directory
==> ERROR: A package has already been built. (use -f to overwrite)
++++++++++++++++

patches directory and this file does not exist.

 

- After the dialog asking for mysql_install_db to be run ...

-- for first time install of mariadb I need to perform
sudo systemctl enable mariadb.service
sudo systemctl start mariadb.service

-- then to set up the root password
mysql_secure_installation

Might be an idea to either suggest in the dialog or to automate (I had no prior exposure to using Mysqlserver/MariaDB).


- Error
Database setup routine fails looping through the following:
+++++++++++++++++
cannot open tty-output
ERROR 1698 (28000): Access denied for user 'root'@'localhost'
+++++++++++++++++

Needed to change the following in ./install.sh:
from:
mysql -u root -e ";"
to:
sudo mysql -u root -e ";"    #### $SUDO is not set on the re-run

# ...as the mysql_install_db states after completion access to root can only be made from the host root user

..and from this all the mysql calls at this point onwards in the script need "sudo" in front of them in the script (bar the one using $mysqlconnectdb which is connecting as the user friendup), ie:
+++++++++++++++++

 Checks if user is already present or not, and creates it eventually
userExists=$(sudo mysql $mysqlAdminConnect \
        --execute="SELECT mu.User FROM mysql.user AS mu WHERE mu.User='$dbuser'")
if [ "$userExists" = "" ]; then
        echo "Setting up user: $dbuser"
        # Creates user
        sudo mysql $mysqlAdminConnect \
                --execute="CREATE USER [email protected]$dbhost IDENTIFIED BY '$dbpass';"
else
        echo "User $dbuser already exists, skipping"
fi

# Checks for database existence and creates it if not present
dbpresent=$(sudo mysql $mysqlAdminConnect \
        --execute="SHOW DATABASES LIKE '$dbname'")
if [ `echo "$dbpresent" | grep -c "$dbname"` -gt 0 ]; then
        echo "Database $dbname was found, skipping"
    # Grants access to db if user was not created before
        sudo mysql $mysqlAdminConnect \
                --execute="GRANT ALL PRIVILEGES ON $dbname.* TO [email protected]$dbhost;"
        # Cleans memory
        sudo mysql $mysqlAdminConnect \
                --execute="FLUSH PRIVILEGES;"
else
        # Creates database
        echo "Creating database: $dbname"
        sudo mysql $mysqlAdminConnect \
                --execute="CREATE DATABASE $dbname"
        # Grants access to db
        sudo mysql $mysqlAdminConnect \
                --execute="GRANT ALL PRIVILEGES ON $dbname.* TO [email protected]$dbhost;"
        # Cleans memory
        sudo mysql $mysqlAdminConnect \
                --execute="FLUSH PRIVILEGES;"
        # Switch to user
        export MYSQL_PWD=$dbpass
        # Creates tables
        echo "Creating tables"
        mysql $mysqlconnectdb \
                --execute="SOURCE db/FriendCoreDatabase.sql"
fi
+++++++++++++++++


Then the script proceeds to compiling the binaries ...

but fails on linking of FriendCore as the following files are missing:
+++++++++++++++++
cc: error: ../libs-ext/openssl/libssl.a: No such file or directory
gcc: error: ../libs-ext/openssl/libcrypto.a: No such file or directory
make[1]: *** [Makefile:201: bin/FriendCore] Error 1
make[1]: Leaving directory '/home/friend/friendup/core'
make: *** [Makefile:157: release] Error 2
+++++++++++++++++

(Of note this was the same on a Debian 9 build I tried as well as an 18.04 Ubuntu build attempt).

Arch does not keep .a files anymore by default in PKGBUILD I read online but these can be built by using PKGBUILD and setting the following option:
options=('staticlibs')

So I tried building an AUR openssl package using this AUR link https://aur.archlinux.org/packages/openssl-git/ to give me these files. It also highlighted during makepkg that it required the binary fakeroot which led to the need for base-devel package (which handily also includes the patch binary that the install script refers to).

However on building and installing my encrypted drive would not start and had a kernel panic as it no longer had the file ...
usr/lib/libssl.so.1.1

**I blame no one here but myself** and thought to relay this as I created the install of Arch to just give the install another go on a different system to see if it worked better but this surprised me...though to the more experienced this may have been obvious : )

On rebuilding my Manjaro Arch install I tried rebuilding this package again (makepkg -s ## no -i for install this time of this openssl-gt AUR package) but without installing it and simply copying the produced libssl.a file into the ../libs-ext/openssl directory (which I admit is a stab in the dark for me, as I am not an experienced developer, and this was just an experiment given I had run out of ideas of how to get around this issue).  On Debian 9 and Ubuntu 18.04 I had to copy these files from /usr/lib/...exact location forgotten... into this directory. Only on one single attempt on Ubuntu however was I able to launch the webclient.

This compilation on Arch failed again with the linking:
+++++++++++++++++
/usr/bin/ld: cannot find -lssh_threads
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:201: bin/FriendCore] Error 1
make[1]: Leaving directory '/home/friend/friendup/core'
make: *** [Makefile:157: release] Error 2
+++++++++++++++++

And at that point I stopped and composed this feedback.

 

compilation.log.firstpass.unsuccessful.txt

Share this post


Link to post
Share on other sites

Hey @amblingbear! Thanks for the feedback. The installer is broken. Sorry about that - we are going to remove it from the repository. The correct way to build Friend now is to install the dependencies and run through the make process;

1. Install dependencies

2. make clean setup

3. make -j compile

4. make install

This should give you a working Friend Core if you have the correct dependencies. Sorry about the inconvenience.

Share this post


Link to post
Share on other sites

Hi @m0ns00n, Thanks for the swift feedback.

TL;DR - I mostly cross-compared files in Debian in the listed dependencies on the github page for FriendUp https://github.com/FriendUPCloud/friendup and identified packages in Arch, found that I was missing libssh_threads.so (which broke linking), but didn't want to trash my encrypted Manjaro build (at least not today 🙂 by using an AUR package that might still have this file, so edited two Makefiles and it compiled and ran (but did not display as expected). Maybe I have not identified all of the dependencies.

If I am better off using Ubuntu then I am happy to do so (as my goal was to look at Friend more than use Arch for it)?

I've relayed my findings in case they are of any help or interest.

 


Details below.

Arch package dependencies (git listed ones for Debian/Ubuntu hashed):
bash
libssh2 # libssh2-1-dev
libssh  # libssh-dev
openssl # libssl-dev
libaio  # libaio-dev
mariadb # mysql-server
mariadb-clients
php   # php7-cli
php-gd   # php7-gd
php-imap # php7-imap
phpmyadmin # php7-mysql
# php7-curl
mariadb-libs  # libmysqlclient-dev
base-devel  # build-essential
libmatheval   # libmatheval-dev
file # libmagic-dev
gd  # libgd-dev
libuv
rsync
valgrind  # valgrind-dbg
libxml2   # libxml2-dev
# php7-readline (php7.0-readline)
cmake
base-devel # ssh
# automake (see ssh)
smbclient # libsmbclient-dev
# from the install script (now defunct) it also mentioned
libwebsockets


make -j compile
ends with ...

/usr/bin/ld: cannot find -lssh_threads
collect2: error: ld returned 1 exit status
make[1]: *** [Makefile:201: bin/FriendCore] Error 1
make[1]: *** Waiting for unfinished jobs....
make[1]: Leaving directory '/home/friend/friendup/core'
make: *** [Makefile:139: compile] Error 2


Googling reveals that libssh_threads.so no longer exists in Arch:
https://github.com/artyom-poptsov/guile-ssh/issues/9
One of the comments here mentions:
"On openSUSE we ran also into this issue. I created a little patch removing -lssh-threads as the required symbols are now provided by libssh.so."
So I wondered if this could be the case in Arch...so I tried removing reference to "USE_SSH_THREADS_LIB=$(USE_SSH_THREADS_LIB)" just to see what would happen...

friendup/Makefile
#       make -C core DEBUG=1 WEBSOCKETS_THREADS=$(WEBSOCKETS_THREADS) USE_SELECT=$(USE_SELECT) NO_VALGRIND=$(NO_VALGRIND
) CYGWIN_BUILD=$(CYGWIN_BUILD) USE_SSH_THREADS_LIB=$(USE_SSH_THREADS_LIB) OPENSSL_INTERNAL=$(OPENSSL_INTERNAL) ENABLE_SS
H=$(ENABLE_SSH)
        make -C core DEBUG=1 WEBSOCKETS_THREADS=$(WEBSOCKETS_THREADS) USE_SELECT=$(USE_SELECT) NO_VALGRIND=$(NO_VALGRIND
) CYGWIN_BUILD=$(CYGWIN_BUILD) OPENSSL_INTERNAL=$(OPENSSL_INTERNAL) ENABLE_SSH=$(ENABLE_SSH)

and also reference to it here:

riendup/core/Makefile
#ifeq ($(USE_SSH_THREADS_LIB),1)
#LFLAGS                                 +=      -lssh_threads
#endif


After this "make install" completed 🙂

However, on running FriendCore I found the following:

Firefox 74.0b9 (64 bit) - login screen comes up but on login with default fadmin this leads to a dark grey blank browser window
Chromium - login screen comes up but it leads to a screen with text and graphic display that is clearly not ok.

 

Share this post


Link to post
Share on other sites

Thanks - I think what we should do is to create "Install Friend OS on X Distro" pages on our to be released documentation site. For now, I think it's best if you run a Virtual Box or similar with Ubuntu 18+.

We have discussed internally to automatically generate images for VMWare and other virtualization software based on our release branches. This will make it much easier.

Have you discussed with the gang on Discord about the Docker image?

Share this post


Link to post
Share on other sites

Some linux distros require ssh_threads lib, some not. So the best idea was to put that into config.

If you want overwrite default configiration settings create Config file. Put it in same place where Config.defs is and all options set in Config file will be used instead of Config.defs.

Share this post


Link to post
Share on other sites

Thanks @m0ns00n and @Pawel. I had trashed my Manjaro desktop and went back to Ubuntu 18.04. I am going to look, as you suggest, look at both Docker and/or VMs, but cannot do so immediately but wanted to say thanks for feedback as silence to posts can be ambiguously interpreted. Thanks again!  

Share this post


Link to post
Share on other sites

TL;DR - Success : )

I created an Ubuntu 18.04 VM using Virtualbox, and was after specifying / adding to the dependencies, editing the existing install.sh script (for the db install section), and copying into place some required .a archive files into libs-ext/openssel, able to get the FriendUp front end running (both on Firefox and Google).

Noted some issues raised about dependency conflicts, and only one error in FriendCore output. All included in the attached build_notes.txt. However, FriendUp GUI was able to load, so happy, and as always hope this feedback helps others in some small way.

 

 

 

build_notes.txt install.sh install.ubuntu18_04.friend.dependencies.sh

Share this post


Link to post
Share on other sites

@m0ns00n@Pawel

Hi,

Back on Manjaro (I'd thought I'd try in VMWare this time and try out the Config file idea relayed) so I tried editing the /home/friend/friendup/Config file with

USE_SSH_THREADS_LIB=0

but this made no difference and the "make -j" step failed. As before I proceeded and edited /home/friend/friendup/core/Makefile instead.

This time however, using the latest version of friendup from git ( even commenting out the section

#ifeq ($(USE_SSH_THREADS_LIB),1)
#LFLAGS                                 +=      -lssh_threads
#endif

made no difference and the build failed with the following errors still:

gcc: error: ../libs-ext/openssl/libssl.a: No such file or directory
gcc: error: ../libs-ext/openssl/libcrypto.a: No such file or directory

 

Now, I am happy with Ubuntu working but thought to relay these findings. The package installation script and its commands attached in case this is useful. Let me know if there is a quick correction that needs to be applied to make this work as I am happy to try it out.

Apologies if I am missing anything obvious.

 

install.manjaro.friend.dependencies.sh

Share this post


Link to post
Share on other sites

Hi @m0ns00n,

Thanks for your reply! I was just writing this response as I succeeded in getting it to compile (but not run the front-end).


Hi @Pawel,

The now defunct install.sh used to create a Config file /home/friend/friendup/Config 
After a new git clone of the friendup repo I created with the Config file with the following entries:

FRIEND_PATH="/home/friend/friendup/build"
USE_SSH_THREADS_LIB=0
OPEN_SSL_INTERNAL=0

So, for me I needed to explicitly switch OPEN_SSL_INTERNAL usage off on the base Manjaro install.

With this it does compile 🙂

(The Makefile under core gave me the idea with its conditional LFLAGS setting and also on Discord #general I noted that there was a post to a US tech doc site which mentioned the same). 


However, on running FriendCore I receive the following: 
 (core/library.c:166) 140539431540672 [LibraryOpen] Library error: /home/friend/friendup/core/bin/libs/mysql.library: cannot open shared object file: No such file or directory  DYNAMIC LINK ERROR

Any ideas / guidance welcome?  

Share this post


Link to post
Share on other sites

Try:

cd "where friendup is"

find . -name mysql.library

This library is created when you are doing compilation. It should be moved to path where FriendCore is installed + path libs/ .

So :

- maybe it was not compiled (some problems) - please give me compilcation output

- or maybe its copyed to wrong path

Share this post


Link to post
Share on other sites

:: make clean setup > make.clean.setup.log 2>&1
## log file attached


:: make -j compile > make.compile.log 2>&1
## log file attached

:: pwd
/home/friend/friendup

:: ls -l libs/mysql/bin/mysql.library
-rwxr-xr-x 1 friend friend 246184 Mar 12 22:16 libs/mysql/bin/mysql.library

:: make install > make.install.log 2>&1
: grep mysql.library make.install.log
cp bin/mysql.library /home/friend/friendup/build/libs/

ls -lrt bin/mysql.library /home/friend/friendup/build/libs/mysql.library

:: pwd
/home/friend/friendup

:: ls -lrt bin/mysql.library /home/friend/friendup/build/libs/mysql.library
ls: cannot access 'bin/mysql.library': No such file or directory
-rwxr-xr-x 1 friend friend 246184 Mar 12 22:19 /home/friend/friendup/build/libs/mysql.library


FriendCore compiles but provides the following output (extract):
:: ./FriendCore
 (main.c:198) 139979829385152 FRIEND_HOME set to: /home/friend/friendup/core/bin
iniparser: cannot open '/home/friend/friendup/core/bin/cfg/cfg.ini'
  (util/log/log.c:254) 139979829385152 [log.c]: Cannot open new file to store logs
...

  (core/library.c:166) 140024704886720 [LibraryOpen] Library error: /home/friend/friendup/core/bin/libs/mysql.library: cannot open shared object file: No such file or directory  DYNAMIC LINK ERROR
  (core/library.c:194) 140024704886720 [LibraryOpen] Library was not possible to open: mysql.library: cannot open shared object file: No such file or directory file mysql.library
  (core/library.c:52) 140024704886720 Lib open
  (core/library.c:89) 140024704886720 [LibraryOpen] Open library /home/friend/friendup/core/bin/mysql.library
  (core/library.c:136) 140024704886720 [LibraryOpen] Open library /home/friend/friendup/core/bin/libs/mysql.library
  (core/library.c:160) 140024704886720 [LibraryOpen] Cannot open file


Given the way I used an edited install.sh (I commened out the compile section) I maybe need to follow the guidance of https://docs.friendup.tech/en-US/docs/Administrators/Installation/Manual_installation and create this file as currently I have cfg.ini in the following location:

With the following content:
; Friend Core configuration file
; ------------------------------
; Please respect both spaces and breaks between lines if you change this file manually
;
[DatabaseUser]
login = "friendup"
password = "friendup1"
host = "localhost"
dbname = "friendup"
port = 3306
 
[FriendCore]
fchost = "localhost"
port = 6502
fcupload = storage/
 
[Core]
port = 6502
SSLEnable = 0
 
[FriendNetwork]
enabled = 0
 
[FriendChat]
enabled = 0


This is different to the content of ~/friendup/docs/cfg.ini.example file for the [DatabaseUser] section:
[DatabaseUser]

host=localhost
login=root
password=root_password
dbname=FriendMaster

[FriendCore]
fchost = localhost
port = 6502
fcupload = storage/

[Core]
SSLEnable=0
port=6502

[FriendNetwork]
enabled = 0

[FriendChat]
enabled = 0


I note that the Readme.md file (referring to the old install.sh script) states the following:
"The installer creates two global variables, FRIEND_HOME pointing to the cloned friendup directory, and FRIEND_PATH pointing to the directory where Friend Core has been build."

I didn't spot this need for FRIEND_HOME in the doc of https://docs.friendup.tech/en-US/docs/Administrators/Installation/Manual_installation so added this to the ~/friendup/Config file,. so tried this:

# Friend Core compilation path
# If empty compilation will default to
# /home/friend/friendup/build
FRIEND_PATH="/home/friend/friendup/build"
FRIEND_HOME="/home/friend/friendup"


I tried
make clean setup
make -j compile
make install

but the FriendCore output shows FRIEND_HOME set elsewhere

[email protected]:~/friendup/core/bin$ ./FriendCore
 (main.c:198) 139800864442176 FRIEND_HOME set to: /home/friend/friendup/core/bin
iniparser: cannot open '/home/friend/friendup/core/bin/cfg/cfg.ini'

Let me know what actions I could take to help or if I have made a mistake int he process or setup.

 

make.clean.setup.log make.compile.log make.install.log

Share this post


Link to post
Share on other sites

OK I assume you are launching FriendCore from core/bin directory.

But you should launch it from friendup/build

You can disable this 2 entries in Config file:

FRIEND_PATH="/home/friend/friendup/build"
FRIEND_HOME="/home/friend/friendup"

Share this post


Link to post
Share on other sites

Thanks @Pawel for seeing the woods from the trees 🙂

I followed your Config directions, used the cfg/cfg.ini that was created by the first half of the old install script (posted previously), rebuilt FriendCore (and launched it from the correct directory)  and had partial success accessing the FriendUP worktop. The Friend login screen came up. On login though the desktop is not as in Ubuntu. Screenshots attached.

The only highlighted terminal output from friendup/build/FriendCore is as follows:

  (core/friend_core.c:1761) 140593448343488 epoll event 1 - listen 62

friend_core log attached.

chromium.74.0.64bit.png

googlechrome.80.0.3987.132.64bit.manjaro.png

friend_core_log-2020-03-13-1.log

Share this post


Link to post
Share on other sites

 Double checked for this package in case I missed it (I listed above what I used on Mar 03), and no such package found:

# setup pacman package search
sudo pacman -Fy

sudo pacman -S php-mysql # nothing

However, looking online I see:
https://wiki.archlinux.org/index.php/PHP#MySQL/MariaDB

So, I enabled in /etc/php/php.ini the two lines:

extension=pdo_mysql
extension=mysqli

and started friendup/build/FriendCore.

Beautiful! - Success :))

Thanks Friends for your help and I hope this helps others.

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


×
×
  • Create New...