Major Windows Install errors

Use this forum for questions regarding installation, upgrading and configuration of OpenEMM

Moderator: moderator

OldManRiver
Posts: 9
Joined: Mon Feb 11, 2008 7:03 pm
Contact:

Major Windows Install errors

Post by OldManRiver »

All,

There are so many errors in the windows install package I do not know where to start. It is a major effort going through the install item by item
and so many obvious problems that I will have to deal with each item as it occurs or not do this justice. First let me say I searched for email addresses and support links to try to report this tyraid of problems but found none so had to register on this board to post the problems.

But rather than just "curse the darkness", I'm the "I'll light a match" type person, so will try to address and correct the issues as I come to them.

Anyway the list of problems are:
  • To the PHP programmer and/or non-python programmer the choice of python as install
    language seems totally wrong, as so much of the install depends on internet install
    packages, which are better called/opened in native PHP,
  • The list of needed addons, which is not well documented on the website, but rather in
    the "README.txt" and Install_Win_OpenEMM-5.3.2.txt obtained with the download.
    These should be clearly labeled under a section on the WebSite on the "Download"
    page of "Required Downloads" so there is no confusion about needed addons,
  • Designed in Security problem - It has been a published security standard, by the Network
    security consortitium, for all Windows installs regardless of OS version, since 1994 that
    all secure installs will follow the minimum of:
    • Minimum of 3 physical or logical drives mapped as follows:
      • C: Contains only the OS and OS registry (nothing else),
      • D: Contains only the Applications/Programs with their supplemental registry info,
      • E: Contains only the Data including a "Move" of the "My Documents" folder for each user to ensure all data is here
        (Alternate to E: is Network drive of H: or other logical drive),
    Since the install does not prompt for install directory, a requirement for all apps developed after June 1994
    "must prompt for install directory", per the security publication, this is a major oversite and design flaw,
  • Therefore based on the last item I had to copy line #49 in the openemm.py file and assign
    "home" the correct install path,
  • Was not sure of the following line (# 73 in my copy)
    • resin = binhome + os.path.sep + 'httpd.exe'
    Seems to cause conflict between installed Apache localhost and the httpd.exe in the \bin directory. But no test to find an installed version of Apache is made,
  • The following code always errors and there is no reference to any DBase IV/V modules anywhere in any of the documents, so what is intended here is totally bogus:

    Code: Select all

    lines 75-82 of my copy of openemm.py
    import  agn
    
    show ('found codebase .. ')
    
    # Check for working database
    if not 'DBase' in dir (agn):
           error ('No database module found')
    
    The database chosen is MySQL so why are calls made to any other DB? Since I've had MySQL installed and working for 3 years there is no problem with the right MySQL DB modules being loaded,
  • Lines 87-95 are suppose to find the JDK in the Windows registry and assign a path, but this is not written right and finds nothing on my system not JDK or JRE or anything Java,
  • Lines 97-99 are suppose to find the MySQL in the Windows registry and assign a path, but this is not written right and finds nothing on my system. I have WAMP installed on both my Windows boxes with upgrades to latest Apache, MySQL PHP but using WAMP directory structure. Obviously this openemm.py install routine does not know how to recognize the correct registry keys for these. I had to add the line:
    • mysqlhome = '\\Program Files\wamp\mysql'
    for this to find mysql,
  • Even with the MySQL path correct the mysql code would not execute correctly. The programmer forgot that not installs of MySQL install the mysqladmin exe as shown in line #119 of my copy with:
    • 119 deflgstr = 'mysqladmin -u root -p create openemm'
    My versions of the MySQL install, installed mysql.exe, mysqld-nt.exe, mysqldump.exe only, so again I had to change code, but these to not process passed vars/args the same, so was forced to open phpMyAdmin and drop the SQL from the openemm.sql file in manually and execute it, to gen the DB. Here again a flaw, for a subsequent pass of the setup routine should check to see if the DB is installed and jump over this portion of install processing when all tables are found,
  • Line #128-269 in the openemm.py file (my copy) reading:
    • 128 db = agn.DBase ()
      <br>
      268 db.close ()
      269 show ('found database.\n')
    Are obviously going to error because of the call to the non-existent and non-documented DBase IV/V module(s) that must be loaded somewhere in one of the configs. I sure would like to see the documentation on what is intended here, because code with no explaination is just not good coding practice. Even if the programmers had followed the directive to document their code some understanding might have been reached going through the code,
  • Basically after that if one comments out all the lines from line #88:
    <ul>
    88 jdkkey = r'SOFTWARE\JavaSoft\Java Development Kit'
to line #285
  • 285 os.chdir (home)
then re-run the setup.bat file, you might actually get an install, but don't hold your breath.

It's unfortunate that the programmers on this are totally deprived of the problems of doing a proper install, so did not anticipate any issues, outside their own sterile and pre-defined environments. Of course that is why there are forums and other help exchanges to document the shortfalls, so they can be addressed.

As promised, "I will light a candle", but since I'm severly challenged in the python arena, this will take me some time, so patience please!

Currently working on the first problem to prompt for an install path/directory.

Thanks!

OMR
Slisserf
Posts: 1
Joined: Tue Mar 04, 2008 5:22 pm

windows version 5.4 not working

Post by Slisserf »

After following all the instructions carefully, the openemm 5.4 windows version has been installed. However after running start.bat it doesn't seem that the mentioned two instances of servlet engine Resin start. Nor does the application start after pointing my local web browser to http://localhost:8081/. There's definitely something wrong. I have no clue how to get the application starting.

I am Looking for a solution and hope someone can help.

Kind regards,

Frank
OldManRiver
Posts: 9
Joined: Mon Feb 11, 2008 7:03 pm
Contact:

Buzy & Distracted

Post by OldManRiver »

All,

I apologize for not getting a solution on this yet!

I've got two major data migration project out there and so haven't had time to address the Python, yet.

I'm going to post on the Python Forum to get help on this, so you can find my posts there and follow along.

Thanks!

OMR
OldManRiver
Posts: 9
Joined: Mon Feb 11, 2008 7:03 pm
Contact:

Status

Post by OldManRiver »

All,

Here's where I'm at in my code:

Code: Select all

#       -*- python -*-
"""**********************************************************************************
* The contents of this file are subject to the Common Public Attribution
* License Version 1.0 (the "License"); you may not use this file except in
* compliance with the License. You may obtain a copy of the License at
* http://www.openemm.org/cpal1.html. The License is based on the Mozilla
* Public License Version 1.1 but Sections 14 and 15 have been added to cover
* use of software over a computer network and provide for limited attribution
* for the Original Developer. In addition, Exhibit A has been modified to be
* consistent with Exhibit B.
* Software distributed under the License is distributed on an "AS IS" basis,
* WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for
* the specific language governing rights and limitations under the License.
*
* The Original Code is OpenEMM.
* The Original Developer is the Initial Developer.
* The Initial Developer of the Original Code is AGNITAS AG. All portions of
* the code written by AGNITAS AG are Copyright (c) 2007 AGNITAS AG. All Rights
* Reserved.
*
* Contributor(s): AGNITAS AG.
**********************************************************************************
"""
#
import  sys, os, time
dir(os)
#
def show (s):
        sys.stderr.write (s)
        sys.stderr.flush ()
def prompt (prmt):
        if prmt:
                show (prmt)
        return sys.stdin.readline ().strip ()
def error (msg):
        show (msg + '\n')
        prompt ('[press return]')
        sys.exit (1)
def addpath (path):
        parts = os.environ['PATH'].split (os.path.pathsep)
        if not path in parts:
                parts.insert (0, path)
                os.environ['PATH'] = os.path.pathsep.join (parts)
#
show ('Starting up .. ')
# try:
homedrive = os.getcwd()
if homedrive is None:
        homedrive = os.environ['HOMEDRIVE']
#except KeyError:
#        homedrive = 'C:'
home = homedrive + os.path.sep + 'OpenEMM'
#show (home)
# home = 'E:\Mail\OpenEMM'
# home = 'D:\Mail\OpenEMM'
#if not os.path.isdir (home):
#        guess = None
#        for disk in 'CDEFGHIJKLMNOPQRSTUVWXYZ':
#                temp = disk + ':' + os.path.sep + 'OpenEMM'
#                if os.path.isdir (temp):
#                        guess = temp
#                        break
#        if guess is None:
#                error (' Failed to find homedir "%s"' % home)
#        home = guess
show ('home is %s .. ' % home)
#
os.environ['HOME'] = home
binhome = home + os.path.sep + 'bin'
show (' binhome is %s .. ' % binhome)
addpath (binhome)
schome = binhome + os.path.sep + 'scripts'
os.environ['PYTHONPATH'] = schome
if not schome in sys.path:
        sys.path.append (schome)
addpath (schome)
os.environ['LC_ALL'] = 'C'
os.environ['LANG'] = 'en_US.ISO8859_1'
os.environ['NLS_LANG'] = 'american_america.UTF8'
resin = binhome + os.path.sep + 'httpd.exe'

#impagn = binhome + os.path.sep + 'agn'
import agn
#dirlist = dir(agn)
#show (dirlist)
show ('found codebase .. ')
#
# Check for working database
#if not 'DBase' in dir (agn):
#       error ('No database module found')
#
# add python to path
addpath (agn.pythonpath)
#
# find jdk
jdkkey = r'SOFTWARE\JavaSoft\Java Development Kit'
version = agn.winregFind (jdkkey, 'CurrentVersion')
#show ('Java Version => ' + version)
#if version is None:
#       error ('JDK not found')
#javahome = agn.winregFind (jdkkey + '\\' + version, 'JavaHome')
javahome = 'C:\Program Files\Java\jre1.6.0_02'
addpath (javahome + os.path.sep + 'bin')
#
# find mysql
#mskey = r'SOFTWARE\MySQL AB\MySQL Server 5.0'
#mysqlhome = agn.winregFind (mskey, 'Location')
mysqlhome = 'E:\Program Files\wamp\mysql'
if not mysqlhome is None:
        addpath (mysqlhome + os.path.sep + 'bin')
#
# build additional CLASSPATH
cp = []
libdir = home + os.path.sep + 'lib'
olibdir = libdir + os.path.sep + 'openemm'
cp = [olibdir]
for fname in os.listdir (olibdir):
        if fname.endswith ('.jar'):
                cp.append (olibdir + os.path.sep + fname)
#
# Optional commands
if len (sys.argv) > 1:
        os.chdir (home)
        if sys.argv[1] == 'setup':
                show ('setup:\n')
#               show ('Setup database, please enter the super user password defined during MySQL instllation:\n')
#                deflgstr = 'mysqladmin -u root -p create openemm'
#                deflgstr = "'E:\Program Files\wamp\mysql\bin\mysql' -u root"
#                if os.system (deflgstr):
#                        error ('Failed to create database')
#                show ('Database created, now setting up initial data, please enter again your databae super user password:\n')
#                if os.system ('mysql -u root -p -e "source USR_SHARE\\openemm.sql" openemm'):
#                        error ('Failed to setup database')
#                show ('Database setup completed.\n')
        if sys.argv[1] in ('setup', 'config'):
                db = agn.DBase ()
                if not db:
                        error ('Failed to setup database connection')
                i = db.newInstance ()
                if not i:
                        error ('Failed to connect to database')
                rdir = None
                mailloop = None
                for r in i.query ('SELECT rdir_domain, mailloop_domain FROM company_tbl WHERE company_id = 1'):
                        rdir = r[0]
                        mailloop = r[1]
                if sys.argv[1] == 'config':
                        show ('config:\n')
                if rdir is None: rdir = ''
                nrdir = prompt ('Enter redirection domain [%s]: ' % rdir)
                if not nrdir: nrdir = rdir
                if mailloop is None: mailloop = ''
                nmailloop = prompt ('Enter mailloop domain [%s]: ' % mailloop)
                if not nmailloop: nmailloop = mailloop
                if nrdir != rdir or nmailloop != mailloop:
                        i.update ('UPDATE company_tbl SET rdir_domain = :rdir, mailloop_domain = :mailloop WHERE company_id = 1',
                                  { 'rdir': nrdir, 'mailloop': nmailloop })
                        db.commit ()
                i.close ()
                db.close ()
                sfname = 'conf' + os.path.sep + 'smart-relay'
                try:
                        fd = open (sfname)
                        sr = fd.read ().strip ()
                        fd.close ()
                except IOError:
                        sr = ''
                show ('Smart mail relay - optional parameter. Specifiy this, if you want to send\n')
                show ('all your outgoing mail via one deticated server (e.g. your ISP mail server.)\n')
                show ('You may add login information in the form <username>:<password>@<relay> if\n')
                show ('the smart relay requires authentication.\n')
                nsr = prompt ('Enter smart relay (or just - to remove existing one) [%s]: ' % sr)
                if nsr:
                        if nsr == '-':
                                try:
                                        os.unlink (sfname)
                                except OSError:
                                        pass
                        elif nsr != sr:
                                fd = open (sfname, 'w')
                                fd.write ('%s\n' % nsr)
                                fd.close ()
                prompt ('Congratulations, %s completed! [return] ' % sys.argv[1])
        elif sys.argv[1] == 'update':
                show ('update:\n')
                db = agn.DBase ()
                if not db:
                        error ('Failed to setup database connection')
                i = db.newInstance ()
                if not i:
                        error ('Failed to connect to database')
                table = '__version_tbl'
                found = False
                for r in i.query ('SHOW TABLES'):
                        if r[0] == table:
                                found = True
                                break
                if not found:
                        version = '5.1.0'
                        tempfile = 'version.sql'
                        fd = open (tempfile, 'w')
                        fd.write ('CREATE TABLE %s (version varchar(50));\n' % table)
                        fd.close ()
                        show ('Database update, please enter your database super user password now\n')
                        st = os.system ('mysql -u root -p -e "source %s" openemm' % tempfile)
                        try:
                                os.unlink (tempfile)
                        except OSError:
                                pass
                        if st:
                                error ('Failed to setup database')
                        i.update ('INSERT INTO %s VALUES (:version)' % table, {'version': version })
                        db.commit ()
                else:
                        version = None
                        for r in i.query ('SELECT version FROM %s' % table):
                                version = r[0]
                        if version is None:
                                error ('Found version table, but no content in table')
                        elif version == '5.1.0':
                                version = '5.1.1'
                        elif version == '5.1.1':
                                version = '5.3.0'
                ans = prompt ('It looks like your previous version is "%s", is this corrent? [no] ' % version)
                if not ans or not ans[0] in 'Yy':
                        error ('Version conflict!')
                curversion = '5.3.2'
                updates = []
                for fname in os.listdir ('USR_SHARE'):
                        if fname.endswith ('.usql'):
                                base = fname[:-5]
                                parts = base.split ('-')
                                if len (parts) == 2:
                                        updates.append ([parts[0], parts[1], 'USR_SHARE\\%s' % fname])
                seen = []
                while version != curversion:
                        found = False
                        for upd in updates:
                                if upd[0] == version and not upd[2] in seen:
                                        try:
                                                fd = open (upd[2])
                                                cont = fd.read ()
                                                fd.close ()
                                                isEmpty = (len (cont) == 0)
                                        except IOError:
                                                isEmpty = False
                                        if not isEmpty:
                                                show ('Database upgrade from %s to %s, please enter your super user password now\n' % (version, upd[1]))
                                                if os.system ('mysql -u root -p -e "source %s" openemm' % upd[2]):
                                                        error ('Failed to update')
                                        else:
                                                show ('No database update from %s to %s required\n' % (version, upd[1]))
                                        version = upd[1]
                                        seen.append (upd[2])
                                        i.update ('UPDATE %s SET version = :version' % table, {'version': version})
                                        db.commit ()
                                        found = True
                                        break
                        if not found:
                                error ('No update from %s to %s found' % (version, curversion))
                i.close ()
                db.close ()
                show ('Update to version %s finished! You may start config.bat now to see\n' % version)
                prompt ('if there are some new things to setup [return] ')
        else:
                error ('Unknown option %s' % sys.argv[1])
        sys.exit (0)

db = agn.DBase ()
if not db:
        error ('Failed to setup database connection')
i = db.newInstance ()
if not i:
        error ('Failed to connect to database')
i.close ()
db.close ()
show ('found database.\n')
#
# remove potential stale files
sessions = os.path.sep.join ([home, 'webapps', 'openemm', 'htdocs', 'WEB-INF', 'sessions'])
fnames = [agn.winstopfile]
if os.path.isdir (sessions):
        for fname in os.listdir (sessions):
                fnames.append (sessions + os.path.sep + fname)
        for fname in fnames:
                try:
                        os.unlink (fname)
#                       show ('Removed stale file %s.\n' % fname)
                except WindowsError:
                        pass
#
# change to home directory
os.chdir (home)
def pystart (cmd):
        args = cmd.split ()
        args.insert (0, agn.pythonbin)
        return os.spawnv (os.P_NOWAIT, args[0], args)

def resinexec (module, what):
        lpath = home + os.path.sep + 'var' + os.path.sep + 'log' + os.path.sep
        lout = lpath + module + '_stdout.log'
        lerr = lpath + module + '_stderr.log'
        cmd = '%s -conf %s%sconf%s%s.conf' % (resin, home, os.path.sep, os.path.sep, module)
        cmd += ' -verbose'
        cmd += ' -jvm-log %s%s.log' % (lpath, module)
        cmd += ' -resin-home %s' % home
        cmd += ' -server %s' % module
#       cmd += ' %s' % what
        args = cmd.split ()
        env = os.environ.copy ()
        env['LANG'] = 'en_US.ISO8859_1'
        env['CLASSPATH'] = os.path.pathsep.join (cp)
        saveout = os.dup (1)
        saveerr = os.dup (2)
        os.close (1)
        os.close (2)
        os.open (lout, os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0666)
        os.open (lerr, os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0666)
        pid = os.spawnve (os.P_NOWAIT, args[0], args, env)
        os.close (1)
        os.close (2)
        os.dup (saveout)
        os.dup (saveerr)
        os.close (saveout)
        os.close (saveerr)
        return pid
        
def resinstart (module):
        return resinexec (module, 'start')
def resinstop (module):
        return resinexec (module, 'stop')
p_upd = pystart (schome + os.path.sep + 'update.py account bounce')
if p_upd == -1:
        error ('Failed to start update process')
p_dst = pystart (schome + os.path.sep + 'pickdist.py')
if p_dst == -1:
        error ('Failed to start pickdist process')
p_bav = pystart (schome + os.path.sep + 'bav-update.py')
if p_bav == -1:
        error ('Failed to start bav-update process')
p_sem = pystart (schome + os.path.sep + 'semu.py')
if p_sem == -1:
        error ('Failed to start semu process')
p_con = resinstart ('console')
if p_con == -1:
        error ('Failed to start console')
time.sleep (2)
p_rdir = resinstart ('redirection')
if p_rdir == -1:
        error ('Failed to start redirection')
#time.sleep (2)
prompt ('Running, press return for termination: ')
show ('Please press the Resin QUIT buttons to terminate java processes.\n')
#resinstop ('redirection')
#time.sleep (2)
#resinstop ('console')
show ('Signal termination to enviroment\n')
open (agn.winstopfile, 'w').close ()
time.sleep (3)
prompt ('Finished, press [return] ')
show ('(window closes on final termination of all processes) ')
time.sleep (2)
You can see I had to put in:

Code: Select all

homedrive = os.getcwd()
to get the current directory and then comment out the testing that followed as it was looking for "X:\OpenEMM" only and would not accept any other directory structure.

Anyway this gets to line 83 then blows. Says it can't find or use the agn module in

Code: Select all

import agn
but the file is there unless it is looking for compiled version of the "agn.py" file, but this is usually not the case, but I also have the "agn.pyc" compiled file, so something is not setting right in the "path" assignments, at least that my assesment.

OMR
OldManRiver
Posts: 9
Joined: Mon Feb 11, 2008 7:03 pm
Contact:

Been awhile

Post by OldManRiver »

All,

It's been a while since I was able to look at this installer problem and the transistion of the source code has gone from ver 5.3.2 to ver 5.5.1, but absolutely no address of or corrections to the installer problem. Code is still looking to install natively on "C:" which is a no-no!

I quess if you are a Linux guru, you don't care how anything works on Windows.

OMR
OldManRiver
Posts: 9
Joined: Mon Feb 11, 2008 7:03 pm
Contact:

Registry Keys

Post by OldManRiver »

All,

Since the installer is looking to read keys from the registry here are mine:

Python Path:

Code: Select all

D:\development\Python\Lib;D:\development\Python\DLLs;D:\development\Python\Lib\lib-tk
Java JDK/SDK

Code: Select all

D:\development\Java SDK\j2sdk1.4.2
C:\Program Files\Java\jre1.6.0_07
C:\Program Files\Java\jre1.6.0_07\bin
Apache

Code: Select all

HKEY_LOCAL_MACHINE/System/ControlSet001/Services/wampapache
ImagePath = "D:\Program Files\Wamp\apache2\bin\Apache.exe" -k runservice
HKEY_LOCAL_MACHINE/System/ControlSet002/Services/wampapache
ImagePath = "D:\Program Files\Wamp\apache2\bin\Apache.exe" -k runservice
MySQL

Code: Select all

HKEY_LOCAL_MACHINE/System/ControlSet001/Services/wampmysqld
ImagePath = ""D:\Program Files\Wamp\mysql\bin\mysqld-nt.exe" "--defaults-file=D:\Program Files\Wamp\mysql\my.ini" wampmysqld
HKEY_LOCAL_MACHINE/System/ControlSet002/Services/wampmysqld
ImagePath = ""D:\Program Files\Wamp\mysql\bin\mysqld-nt.exe" "--defaults-file=D:\Program Files\Wamp\mysql\my.ini" wampmysqld
You can easily see why the current installer has no clue how to install on my machine. Maybe these keys will let the programmers writing the installer package, understand that a search of the Windows registry is required, not just assumed, or pre-assumed locations and keys.

I can not speak for PHPtriad, Xamp, WinAmp or any of the other Windows based AMP installers, but to find apache and mysql on a WAMP installed box the installer software will have to do a wildcard search on the two keys of wampapache and wampmysqld.

An alternate search that would probably work for all is to search the registry, case-insensitve, for apache.exe and my.ini. I would not try the mysql search on the .exe file for mysql, because my experience with the various installers, I think I tried them all and none worked except WAMP, is they all have differently named .exe files.

Anyway I'm hoping my input here, lets the installer programmers know what the issues are and they either understand and correct the issues or they contact me directly, via PM here. I really want to be part of the solution, but my Python knowledge is too limited.

OMR
OldManRiver
Posts: 9
Joined: Mon Feb 11, 2008 7:03 pm
Contact:

Still Erroring

Post by OldManRiver »

All,

Made the following changes to the code:

Code: Select all

#       -*- python -*-
"""**********************************************************************************
* The contents of this file are subject to the Common Public Attribution
* License Version 1.0 (the "License"); you may not use this file except in
* compliance with the License. You may obtain a copy of the License at
* http://www.openemm.org/cpal1.html. The License is based on the Mozilla
* Public License Version 1.1 but Sections 14 and 15 have been added to cover
* use of software over a computer network and provide for limited attribution
* for the Original Developer. In addition, Exhibit A has been modified to be
* consistent with Exhibit B.
* Software distributed under the License is distributed on an "AS IS" basis,
* WITHOUT WARRANTY OF ANY KIND, either express or implied. See the License for
* the specific language governing rights and limitations under the License.
*
* The Original Code is OpenEMM.
* The Original Developer is the Initial Developer.
* The Initial Developer of the Original Code is AGNITAS AG. All portions of
* the code written by AGNITAS AG are Copyright (c) 2007 AGNITAS AG. All Rights
* Reserved.
*
* Contributor(s): AGNITAS AG.
**********************************************************************************
"""
#
import  sys, os, time
#
def show (s):
        sys.stderr.write (s)
        sys.stderr.flush ()
def prompt (prmt):
        if prmt:
                show (prmt)
        return sys.stdin.readline ().strip ()
def error (msg):
        show (msg + '\n')
        prompt ('[press return]')
        sys.exit (1)
def addpath (path):
        parts = os.environ['PATH'].split (os.path.pathsep)
        if not path in parts:
                parts.insert (0, path)
                os.environ['PATH'] = os.path.pathsep.join (parts)
def checkprop (homedir):
        replaces = [
                'jdbc.url=jdbc:mysql://127.0.0.1/openemm',
                'system.script_logdir=var\\log',
                'system.upload_archive=var\\tmp',
                'system.attachment_archive=var\\tmp',
                'log4j.appender.LOGFILE.File=var\\log\\emm_axis.log',
                'log4j.appender.STRUTSLOG.File=var\\log\\emm_struts.log',
                'mailgun.ini.maildir=var\\\\spool\\\\ADMIN',
                'mailgun.ini.metadir=var\\\\spool\\\\META',
                'mailgun.ini.xmlback=bin\\xmlback.exe',
                'mailgun.ini.account_logfile=var\\\\spool\\\\log\\\\account.log'
        ]
        ignores = [
                'system.url',
                'system.updateserver'
        ]
        rplc = {}
        for replace in replaces:
                parts = replace.split ('=', 1)
                rplc[parts[0].strip ()] = replace.replace ('\\', '\\\\') + '\n'
        prop = os.path.sep.join ([homedir, 'webapps', 'core', 'WEB-INF', 'classes', 'emm.properties'])
        save = prop + '.orig'
        fd = open (prop)
        content = fd.readlines ()
        fd.close ()
        ncontent = []
        changed = False
        for line in content:
                if line[0] != '#':
                        parts = line.split ('=', 1)
                        if len (parts) == 2:
                                if rplc.has_key (parts[0]):
                                        nline = rplc[parts[0]]
                                        if nline != line:
                                                line = nline
                                                changed = True
                                elif not parts[0] in ignores:
                                        if '/' in line:
                                                error ('Found possible invalid entry in %s: %s' % (prop, line))
                ncontent.append (line)
        if changed:
                try:
                        os.rename (prop, save)
                except (WindowsError, OSError):
                        pass
                fd = open (prop, 'w')
                fd.write (''.join (ncontent))
                fd.close ()
#
show ('Starting up .. ')
try:
        homedrive = os.environ['HOMEDRIVE']
except KeyError:
        homedrive = 'C:'

###### Changes Start Here ############
#home = homedrive + os.path.sep + 'OpenEMM'
home = 'D:\Mail\OpenEMM'
###### Changes End Here #############

if not os.path.isdir (home):
        guess = None
        for disk in 'CDEFGHIJKLMNOPQRSTUVWXYZ':
                temp = disk + ':' + os.path.sep + 'OpenEMM'
                if os.path.isdir (temp):
                        guess = temp
                        break
        if guess is None:
                error ('Failed to find homedir "%s"' % home)
        home = guess
show ('home is %s .. ' % home)
checkprop (home)
#
os.environ['HOME'] = home
binhome = home + os.path.sep + 'bin'
addpath (binhome)
schome = binhome + os.path.sep + 'scripts'

###### Changes Start Here ############
#os.environ['PYTHONPATH'] = schome
os.environ['PYTHONPATH'] = 'D:\development\Python\Lib'
###### Changes End Here #############

if not schome in sys.path:
        sys.path.append (schome)
os.environ['LC_ALL'] = 'C'
os.environ['LANG'] = 'en_US.ISO8859_1'
os.environ['NLS_LANG'] = 'american_america.UTF8'
resin = binhome + os.path.sep + 'httpd.exe'

import  agn
agn.require ('2.0.0')
show ('found codebase .. ')
#
# Check for working database
if not 'DBase' in dir (agn):
        error ('No database module found')
#
# add python to path
addpath (agn.pythonpath)
#
# find jdk
jdkkey = r'SOFTWARE\JavaSoft\Java Development Kit'
version = agn.winregFind (jdkkey, 'CurrentVersion')
if version is None:
        error ('JDK not found')
javahome = agn.winregFind (jdkkey + '\\' + version, 'JavaHome')
addpath (javahome + os.path.sep + 'bin')
#
# find mysql

###### Changes Start Here ############
#mskey = r'SOFTWARE\MySQL AB\MySQL Server 5.0'
mskey = r'System\ControlSet001\Services\wampapache'
###### Changes End Here #############

mysqlhome = agn.winregFind (mskey, 'Location')
if not mysqlhome is None:
        addpath (mysqlhome + os.path.sep + 'bin')
#
# build additional CLASSPATH
cp = []
#
# Optional commands
if len (sys.argv) > 1:
        os.chdir (home)
        versionTable = '__version_tbl'
        curversion = '5.5.1'
        if sys.argv[1] == 'setup':
                show ('setup:\n')
                show ('Setup database, please enter the super user password defined during MySQL instllation:\n')
                if os.system ('mysqladmin -u root -p create openemm'):
                        error ('Failed to create database')
                show ('Database created, now setting up initial data, please enter again your databae super user password:\n')
                if os.system ('mysql -u root -p -e "source USR_SHARE\\openemm.sql" openemm'):
                        error ('Failed to setup database')
                show ('Database setup completed.\n')
                db = agn.DBase ()
                if not db is None:
                        cursor = db.cursor ()
                        if not cursor is None:
                                cursor.execute ('CREATE TABLE %s (version varchar(50))' % versionTable)
                                cursor.execute ('INSERT INTO %s VALUES (:version)' % versionTable, {'version': curversion})
                                cursor.sync ()
                                cursor.close ()
                        db.close ()
        if sys.argv[1] in ('setup', 'config'):
                db = agn.DBase ()
                if not db:
                        error ('Failed to setup database connection')
                i = db.cursor ()
                if not i:
                        error ('Failed to connect to database')
                rdir = None
                mailloop = None
                for r in i.query ('SELECT rdir_domain, mailloop_domain FROM company_tbl WHERE company_id = 1'):
                        rdir = r[0]
                        mailloop = r[1]
                if sys.argv[1] == 'config':
                        show ('config:\n')
                if rdir is None: rdir = ''
                nrdir = prompt ('Enter redirection domain [%s]: ' % rdir)
                if not nrdir: nrdir = rdir
                if mailloop is None: mailloop = ''
                nmailloop = prompt ('Enter mailloop domain [%s]: ' % mailloop)
                if not nmailloop: nmailloop = mailloop
                if nrdir != rdir or nmailloop != mailloop:
                        i.update ('UPDATE company_tbl SET rdir_domain = :rdir, mailloop_domain = :mailloop WHERE company_id = 1',
                                  { 'rdir': nrdir, 'mailloop': nmailloop })
                        db.commit ()
                i.close ()
                db.close ()
                sfname = 'conf' + os.path.sep + 'smart-relay'
                try:
                        fd = open (sfname)
                        sr = fd.read ().strip ()
                        fd.close ()
                except IOError:
                        sr = ''
                show ('Smart mail relay - optional parameter. Specifiy this, if you want to send\n')
                show ('all your outgoing mail via one deticated server (e.g. your ISP mail server.)\n')
                show ('You may add login information in the form <username>:<password>@<relay> if\n')
                show ('the smart relay requires authentication.\n')
                nsr = prompt ('Enter smart relay (or just - to remove existing one) [%s]: ' % sr)
                if nsr:
                        if nsr == '-':
                                try:
                                        os.unlink (sfname)
                                except (WindowsError, OSError):
                                        pass
                        elif nsr != sr:
                                fd = open (sfname, 'w')
                                fd.write ('%s\n' % nsr)
                                fd.close ()
                prompt ('Congratulations, %s completed! [return] ' % sys.argv[1])
        elif sys.argv[1] == 'update':
                show ('update:\n')
                db = agn.DBase ()
                if not db:
                        error ('Failed to setup database connection')
                i = db.cursor ()
                if not i:
                        error ('Failed to connect to database')
                found = False
                for r in i.query ('SHOW TABLES'):
                        if r[0] == versionTable:
                                found = True
                                break
                if not found:
                        version = '5.1.0'
                        tempfile = 'version.sql'
                        fd = open (tempfile, 'w')
                        fd.write ('CREATE TABLE %s (version varchar(50));\n' % versionTable)
                        fd.close ()
                        show ('Database update, please enter your database super user password now\n')
                        st = os.system ('mysql -u root -p -e "source %s" openemm' % tempfile)
                        try:
                                os.unlink (tempfile)
                        except (WindowsError, OSError):
                                pass
                        if st:
                                error ('Failed to setup database')
                        i.update ('INSERT INTO %s VALUES (:version)' % versionTable, {'version': version })
                        db.commit ()
                else:
                        version = None
                        for r in i.query ('SELECT version FROM %s' % versionTable):
                                version = r[0]
                        if version is None:
                                error ('Found version table, but no content in table')
                        elif version == '5.1.0':
                                version = '5.1.1'
                        elif version == '5.1.1':
                                version = '5.3.0'
                ans = prompt ('It looks like your previous version is "%s", is this corrent? [no] ' % version)
                if not ans or not ans[0] in 'Yy':
                        error ('Version conflict!')
                updates = []
                for fname in os.listdir ('USR_SHARE'):
                        if fname.endswith ('.usql'):
                                base = fname[:-5]
                                parts = base.split ('-')
                                if len (parts) == 2:
                                        updates.append ([parts[0], parts[1], 'USR_SHARE\\%s' % fname])
                seen = []
                while version != curversion:
                        found = False
                        oldversion = version
                        for upd in updates:
                                if upd[0] == version and not upd[2] in seen:
                                        try:
                                                fd = open (upd[2])
                                                cont = fd.read ()
                                                fd.close ()
                                                isEmpty = (len (cont) == 0)
                                        except IOError:
                                                isEmpty = False
                                        if not isEmpty:
                                                show ('Database upgrade from %s to %s, please enter your super user password now\n' % (version, upd[1]))
                                                if os.system ('mysql -u root -p -e "source %s" openemm' % upd[2]):
                                                        error ('Failed to update')
                                        else:
                                                show ('No database update from %s to %s required\n' % (version, upd[1]))
                                        version = upd[1]
                                        seen.append (upd[2])
                                        i.update ('UPDATE %s SET version = :version' % versionTable, {'version': version})
                                        db.commit ()
                                        found = True
                                        break
                        if not found:
                                error ('No update from %s to %s found' % (version, curversion))
                        if oldversion == '5.3.2':
                                i.close ()
                                db.close ()
                                dbfile = os.path.sep.join (['var', 'tmp', 'openemm.dump'])
                                dbconv = os.path.sep.join (['var', 'tmp', 'openemm.conv'])
                                show ('===========================================================================\n')
                                show ('!! Please read and follow the next steps carefully to avoid loss of data !!\n')
                                show ('Now forcing cleanup of the database, please enter your super user password now\n')
                                if os.system ('mysqldump -aCceQ --lock-all-tables -u root -p -r %s openemm' % dbfile):
                                        error ('Failed to dump current database')
                                try:
                                        fdi = open (dbfile, 'r')
                                        fdo = open (dbconv, 'w')
                                except IOError, e:
                                        error ('Failed to open database convertion file %s %s' % (dbconv, `e.args`))
                                fdo.write ('ALTER DATABASE openemm DEFAULT CHARACTER SET utf8;\n')
                                for line in fdi.readlines ():
                                        line = line.replace (' character set utf8 collate utf8_unicode_ci', '')
                                        line = line.replace (' collate utf8_unicode_ci', '')
                                        line = line.replace ('DEFAULT CHARSET=latin1', 'DEFAULT CHARSET=utf8 COLLATE=utf8_unicode_ci')
                                        fdo.write (line)
                                fdo.close ()
                                fdi.close ()
                                show ('Now we remove and recreate the database and import the converted content.\n')
                                show ('Please enter your super user database password each time, if asked for:\n')
                                state = 0
                                while state < 3:
                                        if state == 0:
                                                action = 'Drop database'
                                                command = 'mysqladmin -u root -p drop openemm'
                                        elif state == 1:
                                                action = 'Create database'
                                                command = 'mysqladmin -u root -p create openemm'
                                        elif state == 2:
                                                action = 'Import database'
                                                command = 'mysql -u root -p openemm < %s' % dbconv
                                        show ('--> %s:\n' % action)
                                        if os.system (command):
                                                show ('Command failed! If you have just mistyped your password, just try\n')
                                                show ('again, otherwise abort the update and fix the problem by hand.\n')
                                        else:
                                                state += 1
                                try:
                                        os.unlink (dbconv)
                                except (WindowsError, OSError):
                                        pass
                                db = agn.DBase ()
                                i = db.cursor ()
                                show ('===========================================================================\n')
                i.close ()
                db.close ()
                show ('Update to version %s finished! You may start config.bat now to see\n' % version)
                prompt ('if there are some new things to setup [return] ')
        else:
                error ('Unknown option %s' % sys.argv[1])
        sys.exit (0)

db = agn.DBase ()
if not db:
        error ('Failed to setup database connection')
i = db.cursor ()
if not i:
        error ('Failed to connect to database')
i.close ()
db.close ()
show ('found database.\n')
#
# remove potential stale files
sessions = os.path.sep.join ([home, 'webapps', 'core', 'WEB-INF', 'sessions'])
fnames = [agn.winstopfile]
if os.path.isdir (sessions):
        for fname in os.listdir (sessions):
                fnames.append (sessions + os.path.sep + fname)
        for fname in fnames:
                try:
                        os.unlink (fname)
#                       show ('Removed stale file %s.\n' % fname)
                except (WindowsError, OSError):
                        pass
#
# change to home directory
os.chdir (home)
def pystart (cmd):
        args = cmd.split ()
        args.insert (0, agn.pythonbin)
        return os.spawnv (os.P_NOWAIT, args[0], args)

def resinexec (module, what):
        lpath = home + os.path.sep + 'var' + os.path.sep + 'log' + os.path.sep
        lout = lpath + module + '_stdout.log'
        lerr = lpath + module + '_stderr.log'
        cmd = '%s -conf %s%sconf%s%s.conf' % (resin, home, os.path.sep, os.path.sep, module)
        cmd += ' -verbose'
        cmd += ' -jvm-log %s%s.log' % (lpath, module)
        cmd += ' -resin-home %s' % home
        cmd += ' -server %s' % module
#       cmd += ' %s' % what
        args = cmd.split ()
        env = os.environ.copy ()
        env['LANG'] = 'en_US.ISO8859_1'
        env['CLASSPATH'] = os.path.pathsep.join (cp)
        saveout = os.dup (1)
        saveerr = os.dup (2)
        os.close (1)
        os.close (2)
        os.open (lout, os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0666)
        os.open (lerr, os.O_WRONLY | os.O_APPEND | os.O_CREAT, 0666)
        pid = os.spawnve (os.P_NOWAIT, args[0], args, env)
        os.close (1)
        os.close (2)
        os.dup (saveout)
        os.dup (saveerr)
        os.close (saveout)
        os.close (saveerr)
        return pid

def resinstart (module):
        return resinexec (module, 'start')
def resinstop (module):
        return resinexec (module, 'stop')
os.system ('bin\\xmlback.exe -D > var\\spool\\META\\blockmail.dtd')
p_upd = pystart (schome + os.path.sep + 'update.py account bounce')
if p_upd == -1:
        error ('Failed to start update process')
p_dst = pystart (schome + os.path.sep + 'pickdist.py')
if p_dst == -1:
        error ('Failed to start pickdist process')
p_bav = pystart (schome + os.path.sep + 'bav-update.py')
if p_bav == -1:
        error ('Failed to start bav-update process')
p_sem = pystart (schome + os.path.sep + 'semu.py')
if p_sem == -1:
        error ('Failed to start semu process')
p_con = resinstart ('core')
if p_con == -1:
        error ('Failed to start core')
prompt ('Running, press return for termination: ')
show ('Please press the Resin QUIT button to terminate java process.\n')
#resinstop ('redirection')
#time.sleep (2)
#resinstop ('console')
show ('Signal termination to enviroment\n')
open (agn.winstopfile, 'w').close ()
time.sleep (3)
prompt ('Finished, press [return] ')
show ('(window closes on final termination of all processes) ')
time.sleep (2)
Then ran setup.bat and got these errors:

Code: Select all

E:\CDs\PHP, CSS & Web Stuff\openEMM\Ver-5.5.1\unpack>setup
Starting up .. home is D:\Mail\OpenEMM .. Traceback (most recent call last):
  File "E:\CDs\PHP, CSS & Web Stuff\openEMM\Ver-5.5.1\unpack\bin\openemm.py", li
ne 111, in <module>
    checkprop (home)
  File "E:\CDs\PHP, CSS & Web Stuff\openEMM\Ver-5.5.1\unpack\bin\openemm.py", li
ne 66, in checkprop
    fd = open (prop)
IOError: [Errno 2] No such file or directory: 'D:\\Mail\\OpenEMM\\webapps\\core\
\WEB-INF\\classes\\emm.properties'
But the file exists in the directory:

Code: Select all

E:\CDs\PHP, CSS & Web Stuff\openEMM\Ver-5.5.1\unpack\webapps\core\WEB-INF\classes
which is the install path directory. Sorry not finding a var for this to set it to make sure the install path is properly preserved.

OMR
jacanon7
Posts: 2
Joined: Tue Jul 28, 2009 10:02 pm

Windows Install error

Post by jacanon7 »

Hello Mr. "OldManRiver"

My name is Jorge Cañón, from Ibagué - Colombia

I got this error while OpenEMM installation :( :

Starting up .. home is C:\OpenEMM .. found codebase .. Failed to connect to database
[press return]

I saw your post, but I don't Know exactrly wich file to change.

I'm using:
- Microsoft Windows XP Professional Vesión 2002 Service Pack 3
- Xamp 1.7.0 / mysql-5.1.36-win32.msi
- C:\Archivos de programa\Java\jdk1.6.0_12
- python-2.5.4.msi
- MySQL-python-1.2.2.win32-py2.5.exe
- OpenEMM-5.5.1-win32.zip

Regards
Jorge Andrés Cañón Sierra
Ingeniero de Sistemas
OldManRiver
Posts: 9
Joined: Mon Feb 11, 2008 7:03 pm
Contact:

Where to look

Post by OldManRiver »

J,

You are using Xamp, which does not install the same as if you installed Apache, MySQL and PHP natively, which installed natively is lots of trouble, but the Python installer looks both on the drive and in your registry only for native install paths/locations.

I've worked on the Python, some, but since I'm neither a C++ nor Python programmer, this has not been easy for me. I still have not gotten a successful install of this product, but slowly working toward it.

If you join me I can share what I have so far.

First I had to dig through the Python to find every reference to location and path from the disk locations, and change to actual locations on my drive. Since this is hard coded, I simply copied the existing line, commenting out the old and change the copy to correct location on my drive. You see this in my code. You also see the errors I get. Not sure why.

The registry part is where I run into trouble as I know nothing about registry and it's manipulation. You see my one attempt on that, but since code is still erroring in the path/loction area, not sure if this is correct and going to run.

Since the economic downturn, I had to lay off everyone, so way too busy to even visit this again in over 9 months.

OMR
Post Reply