TouchPortalAPI v1.7.8

TouchPortalAPI.tppbuild

TouchPortal Python TPP build tool

Features

This SDK tools makes compile, packaging and distribution of your plugin easier.

These are the steps the tppbuild will do for you:

  • Generate entry.tp if you passed in .py file otherwise it will validate the .tp file and raise an error if it's not valid.
  • Compile your main script for your system (Windows, MacOS) depending on the platform you're running on.
  • Create a .tpp file with all the files include compiled script, (generated or existing) entry.tp file.
  • Also the .tpp file will be renamed into this format pluginname_version_os.tpp

Note that running this script requires pyinstaller to be installed. You can install it by running pip install pyinstaller in your terminal.

Using it in example

tppbuild --target example_build.py

In this example we targed the example_build.py file because that file contains infomations on how to build the plugin.

Command-line Usage

The script command is tppbuild when the TouchPortalAPI is installed (via pip or setup), or tppbuild.py when run directly from this source.

<script-command> [-h] <target>

Script to automatically compile a Python plugin into a standalone exe, generate entry.tp, and package them into
importable tpp file.

positional arguments:
  <target>    A build script that contains some infomations about the plugin. Using given infomation about the plugin,
              this script will automatically build entry.tp (if given file is .py) and it will build the distro based
              on which operating system you're using.

options:
  -h, --help  show this help message and exit
  1""" 
  2# TouchPortal Python TPP build tool
  3
  4## Features
  5
  6This SDK tools makes compile, packaging and distribution of your plugin easier.
  7
  8These are the steps the tppbuild will do for you:
  9- Generate entry.tp if you passed in .py file otherwise it will validate the .tp file and raise an error if it's not valid.
 10- Compile your main script for your system (Windows, MacOS) depending on the platform you're running on.
 11- Create a .tpp file with all the files include compiled script, (generated or existing) entry.tp file.
 12- Also the .tpp file will be renamed into this format pluginname_version_os.tpp
 13
 14Note that running this script requires `pyinstaller` to be installed. You can install it by running `pip install pyinstaller` in your terminal.
 15
 16Using it in [example](https://github.com/KillerBOSS2019/TouchPortal-API/tree/main/examples)
 17
 18```
 19tppbuild --target example_build.py
 20```
 21In this example we targed the example_build.py file because that file contains infomations on how to build the plugin.
 22
 23## Command-line Usage
 24The script command is `tppbuild` when the TouchPortalAPI is installed (via pip or setup), or `tppbuild.py` when run directly from this source.
 25
 26```
 27<script-command> [-h] <target>
 28
 29Script to automatically compile a Python plugin into a standalone exe, generate entry.tp, and package them into
 30importable tpp file.
 31
 32positional arguments:
 33  <target>    A build script that contains some infomations about the plugin. Using given infomation about the plugin,
 34              this script will automatically build entry.tp (if given file is .py) and it will build the distro based
 35              on which operating system you're using.
 36
 37options:
 38  -h, --help  show this help message and exit
 39```
 40"""
 41
 42__copyright__ = """
 43This file is part of the TouchPortal-API project.
 44Copyright TouchPortal-API Developers
 45Copyright (c) 2021 DamienS
 46All rights reserved.
 47
 48This program is free software: you can redistribute it and/or modify
 49it under the terms of the GNU General Public License as published by
 50the Free Software Foundation, either version 3 of the License, or
 51(at your option) any later version.
 52
 53This program is distributed in the hope that it will be useful,
 54but WITHOUT ANY WARRANTY; without even the implied warranty of
 55MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
 56GNU General Public License for more details.
 57
 58You should have received a copy of the GNU General Public License
 59along with this program.  If not, see <https://www.gnu.org/licenses/>.
 60"""
 61
 62__all__ = ['PLUGIN_MAIN', 'PLUGIN_EXE_NAME', 'PLUGIN_EXE_ICON', 'PLUGIN_ENTRY', 'PLUGIN_ENTRY_INDENT', 'PLUGIN_ROOT',
 63 'PLUGIN_ICON', 'OUTPUT_PATH', 'PLUGIN_VERSION', 'ADDITIONAL_FILES', 'ADDITIONAL_PYINSTALLER_ARGS', 'validateBuild', 'runBuild']
 64
 65import importlib
 66import os
 67import sys
 68from zipfile import (ZipFile, ZIP_DEFLATED)
 69from argparse import ArgumentParser
 70from glob import glob
 71from shutil import rmtree
 72from pathlib import Path
 73try:
 74	import PyInstaller.__main__
 75except ImportError:
 76	print("PyInstaller is not installed. Please install it before running this script.")
 77	sys.exit(1)
 78
 79sys.path.insert(0, os.path.dirname(os.path.realpath(__file__)))
 80import sdk_tools
 81
 82def getInfoFromBuildScript(script:str):
 83	try:
 84		sys.path.insert(1, os.getcwd()) # This allows build config to import stuff
 85		spec = importlib.util.spec_from_file_location("buildScript", script)
 86		buildScript = importlib.util.module_from_spec(spec)
 87		spec.loader.exec_module(buildScript)
 88	except Exception as e:
 89		raise ImportError(f"ERROR while trying to import plugin code from '{script}': {repr(e)}")
 90	return buildScript
 91
 92def build_tpp(zip_name, tpp_pack_list):
 93	print("Creating archive: " + zip_name)
 94	with ZipFile(zip_name, "w", ZIP_DEFLATED) as zf:
 95		for src, dest in tpp_pack_list.items():
 96			zf.write(src, dest + os.path.basename(src))
 97	print("")
 98
 99def zip_dir(zf, path, base_path="./", recurse=True):
100	relroot = os.path.abspath(os.path.join(path, os.pardir))
101	for root, _, files in os.walk(path):
102		for file in files:
103			src = os.path.join(root, file)
104			if os.path.isfile(src):
105				dst = os.path.join(base_path, os.path.relpath(root, relroot), file)
106				zf.write(src, dst)
107			elif recurse and os.path.isdir(src):
108				zip_dir(zf, src, base_path)
109
110def build_distro(opsys, version, pluginname, packingList, output):
111	if opsys == OS_WIN:
112		os_name = "Windows"
113	elif opsys == OS_MAC:
114		os_name = "MacOS"
115	elif opsys == OS_LINUX:
116		os_name = "Linux"
117	else:
118		raise ValueError("Unknown OS")
119	zip_name = pluginname + "_v" + str(version) + "_" + os_name + ".tpp"
120	print("Creating archive: "+ zip_name)
121	if not os.path.exists(output):
122		os.makedirs(output)
123	with ZipFile(os.path.join(output, zip_name), "w", ZIP_DEFLATED) as zf:
124		for src, dest in packingList.items():
125			if os.path.isdir(src):
126				zip_dir(zf, src, dest)
127			elif os.path.isfile(src):
128				zf.write(src, dest + os.path.basename(src))
129
130	print("")
131
132def build_clean(distPath, dirPath=None):
133	print("Cleaning up...")
134	files = glob(distPath)
135	files.extend(glob("__pycache__"))
136
137	for file in files:
138		if os.path.exists(file):
139			print("removing: " + file)
140			if os.path.isfile(file):
141				os.remove(file)
142			elif os.path.isdir(file):
143				rmtree(file)
144	print("")
145
146def filePath(*file):
147	fullpath = os.path.join(*file)
148	return str(Path(fullpath).resolve())
149
150		
151
152EXE_SFX = ".exe" if sys.platform == "win32" else ""
153
154OS_WIN = 1
155OS_MAC = 2
156OS_LINUX = 3
157
158requiredVar = [
159        "PLUGIN_ENTRY", "PLUGIN_MAIN", "PLUGIN_ROOT", "PLUGIN_EXE_NAME"
160    ]
161optionalVar = [
162	"ADDITIONAL_PYINSTALLER_ARGS", "PLUGIN_ICON", "PLUGIN_EXE_ICON",
163	"ADDITIONAL_FILES", "OUTPUT_PATH", "PLUGIN_VERSION", "PLUGIN_ENTRY_INDENT",
164	"ADDITINAL_TPPSDK_ARGS"
165]
166attri_list = requiredVar + optionalVar
167
168def main(buildArgs=None):
169	if sys.platform == "win32":
170		opsys = OS_WIN
171	elif sys.platform == "darwin":
172		opsys = OS_MAC
173	elif sys.platform == "linux":
174		opsys = OS_LINUX
175	else:
176		return "Unsupported OS: " + sys.platform
177
178	parser = ArgumentParser(description=
179		"Script to automatically compile a Python plugin into a standalone exe, generate entry.tp, and package them into importable tpp file."
180	)
181
182	parser.add_argument(
183		"target", metavar='<target>', type=str,
184		help='A build script that contains some infomations about the plugin. ' +
185		'Using given infomation about the plugin, this script will automatically build entry.tp (if given file is .py) and it will build the distro ' +
186		'based on which operating system you\'re using.'
187	)
188
189	opts = parser.parse_args(buildArgs)
190	del parser
191
192	out_dir = os.path.dirname(opts.target)
193
194	if out_dir:
195		os.chdir(out_dir)
196
197	print("tppbuild started with target: " + opts.target)
198	buildfile = getInfoFromBuildScript(os.path.basename(opts.target))
199
200	checklist = [attri in dir(buildfile) for attri in attri_list]
201	if all(checklist) == False:
202		print(f"{opts.target} is missing these variables: ", " ".join([attri for attri in attri_list if attri not in dir(buildfile)]))
203		return -1
204
205	TPP_PACK_LIST = {}
206
207	print(f"Building {buildfile.PLUGIN_EXE_NAME} v{buildfile.PLUGIN_VERSION} target(s) on {sys.platform}\n")
208
209	#buildfiledir = filePath(out_dir)
210
211	if os.path.exists(dirPath := os.path.join(buildfile.OUTPUT_PATH, "dist")):
212		rmtree(dirPath)
213	os.makedirs(dirPath)
214
215	distdir = os.path.join(buildfile.OUTPUT_PATH, 'dist')
216
217	if (entry_abs_path := buildfile.PLUGIN_ENTRY) and os.path.isfile(entry_abs_path):
218		sys.path.append(os.path.dirname(os.path.realpath(entry_abs_path)))
219		entry_output_path = os.path.join(distdir, "entry.tp")
220		if buildfile.PLUGIN_ENTRY.endswith(".py"):
221			sdk_arg = [entry_abs_path, f"-i={buildfile.PLUGIN_ENTRY_INDENT}", f"-o={entry_output_path}"]
222			sdk_arg.extend(buildfile.ADDITINAL_TPPSDK_ARGS)
223		else:
224			sdk_arg = [entry_abs_path, "-v"]
225			entry_output_path = buildfile.PLUGIN_ENTRY
226
227		result = sdk_tools.main(sdk_arg)
228		if result == 0:
229			print("Adding entry.tp to packing list.")
230			TPP_PACK_LIST[entry_output_path] = buildfile.PLUGIN_ROOT + "/"
231		else:
232			print("Cannot contiune because entry.tp is invalid. Please check the error message above. and try again.")
233			return 0
234	else:
235		print(f"Warning could not find {buildfile.PLUGIN_ENTRY}. Canceling build process.")
236		return 0
237	
238	if not os.path.exists(buildfile.PLUGIN_ICON):
239		print(f"Warning {buildfile.PLUGIN_ICON} does not exist. TouchPortal will use default plugin icon.")
240	else:
241		print(f"Found {buildfile.PLUGIN_ICON} adding it to packing list.")
242		TPP_PACK_LIST[buildfile.PLUGIN_ICON.split("/")[-1]] = buildfile.PLUGIN_ROOT + "/" \
243			 if len(buildfile.PLUGIN_ICON.split("/")) == 1 else "".join(buildfile.PLUGIN_ICON.split("/")[0:-1])
244
245	print(f"Compiling {buildfile.PLUGIN_MAIN} for {sys.platform}")
246
247	PI_RUN = [buildfile.PLUGIN_MAIN]
248	PI_RUN.append(f'--distpath={distdir}')
249	PI_RUN.append(f'--onefile')
250	PI_RUN.append("--clean")
251	if (buildfile.PLUGIN_EXE_NAME == ""):
252		PI_RUN.append(f'--name={os.path.splitext(os.path.basename(buildfile.PLUGIN_MAIN))[0]}')
253	else:
254		PI_RUN.append(f'--name={buildfile.PLUGIN_EXE_NAME}')
255	if buildfile.PLUGIN_EXE_ICON and os.path.isfile(buildfile.PLUGIN_EXE_ICON):
256		PI_RUN.append(f"--icon={Path(buildfile.PLUGIN_EXE_ICON).resolve()}")
257
258	PI_RUN.append(f"--specpath={distdir}")
259	PI_RUN.append(f"--workpath={distdir}/build")
260
261	PI_RUN.extend(buildfile.ADDITIONAL_PYINSTALLER_ARGS)
262
263	print("Running pyinstaller with arguments:", " ".join(PI_RUN))
264	PyInstaller.__main__.run(PI_RUN)
265	print(f"Done compiling. adding to packing list:", buildfile.PLUGIN_EXE_NAME + EXE_SFX)
266	TPP_PACK_LIST[filePath(distdir, buildfile.PLUGIN_EXE_NAME + EXE_SFX)] = buildfile.PLUGIN_ROOT + "/"
267	print("Checking for any additional required files")
268	for file in buildfile.ADDITIONAL_FILES:
269		print(f"Adding {file} to plugin")
270		TPP_PACK_LIST[os.path.basename(file)] = os.path.join(buildfile.PLUGIN_ROOT, os.path.split(file)[0])
271
272	print("Packing everything into tpp file")
273	build_distro(opsys, buildfile.PLUGIN_VERSION, buildfile.PLUGIN_EXE_NAME, TPP_PACK_LIST, buildfile.OUTPUT_PATH)
274
275	build_clean(distdir)
276	print("Done!")
277
278	return 0
279
280if __name__ == "__main__":
281	sys.exit(main())
282
283PLUGIN_MAIN = " "
284"""
285*REQUIRED*
286PLUGIN_MAIN: This lets tppbuild know where your main python plugin file is located so it will know which file to compile.
287Note: This can be ether relative or absolute to the main script.
288"""
289
290PLUGIN_EXE_NAME = " "
291"""
292*REQUIRED*
293PLUGIN_EXE_NAME: This defines what you want your plugin executable to be named. tppbuild will also use this for the .tpp file in the format:
294                `pluginname + "_v" + version + "_" + os_name + ".tpp"`
295"""
296
297PLUGIN_EXE_ICON = r" "
298"""
299*OPTIONAL*
300PLUGIN_EXE_ICON: This should be a path to a .ico file. However if png is passed in, it will tries to automatically converted to ico. if `PILLOW` is installed.
301"""
302
303PLUGIN_ENTRY = " "
304"""
305*REQUIRED*
306PLUGIN_ENTRY: This can be either path to entry.tp or path to a python file that contains infomation about entry.
307Note if you pass in a entry.tp, tppbuild will automatically validate the json. If you pass in a python file, it will
308build entry.tp & validate it for you. If validation fails, tppbuild will exit.
309"""
310
311PLUGIN_ENTRY_INDENT = 2
312"""
313*OPTIONAL*
314This allows you to set indent for the `entry.tp` json data. Default is `2`
315but if you want to save space use `-1` meaning no indent. This is only used if `PLUGIN_ENTRY` is a py file.
316"""
317
318PLUGIN_ROOT = " "
319""" 
320*REQUIRED*
321This is the root folder name that will be inside of .tpp
322 """
323
324
325PLUGIN_ICON = r" "
326""" 
327*OPTIONAL*
328Path to icon file that is used in entry.tp for category `imagepath`, if any.
329 If left blank, TP will use a default icon. """
330
331OUTPUT_PATH = r"./"
332"""
333*OPTIONAL*
334This tells tppbuild where you want finished build tpp to be saved at. 
335If leaved empty it will store tpp to current dir where this build config is located. 
336"""
337
338
339PLUGIN_VERSION = " "
340"""
341*OPTIONAL*
342A version string will be used as part of .tpp file. example input `PLUGIN_VERSION = "1.0.0-beta1"`
343"""
344
345ADDITIONAL_FILES = []
346"""
347*OPTIONAL*
348If you have any required file(s) that your plugin needs, put them in this list.
349"""
350
351ADDITIONAL_PYINSTALLER_ARGS = []
352"""
353*OPTIONAL*
354Any additional arguments to be passed to Pyinstaller.
355"""
356
357ADDITIONAL_TPPSDK_ARGS = []
358"""
359*OPTIONAL*
360ADDITIONAL_TPPSDK_ARGS: This allows you to give additional arg when generating entry.tp
361"""
362
363import inspect
364
365def validateBuild():
366    """
367    validateBuild() this takes no arguments. when It's called it will check for all
368    required constants variable are vaild It will check if path is vaild, is a file etc... If 
369    any error is found, It will list all the error during the process.
370    """
371    frame = inspect.stack()[1]
372    module = inspect.getmodule(frame[0])
373    constants = module.__dir__()
374    os.chdir(os.path.split(module.__file__)[0])
375
376    checklist = [attri in constants for attri in attri_list]
377    print("Checking if all constants variable exists")
378    if all(checklist) == False:
379        print(f"{os.path.basename(module.__file__)} is missing these variables: ", " ".join([attri for attri in attri_list if attri not in dir(constants)]))
380        return 0
381
382    print("Checking variable is vaild")
383
384    anyError = False
385
386    if module.PLUGIN_MAIN and not os.path.isfile(module.PLUGIN_MAIN):
387        print(f"PLUGIN_MAIN is ether empty or invalid file path.")
388        anyError = True
389
390    if module.PLUGIN_ENTRY and not os.path.isfile(module.PLUGIN_ENTRY):
391        print(f"PLUGIN_ENTRY is ether empty or invalid file path.")
392        anyError = True
393
394    if not module.PLUGIN_ROOT:
395        print("PLUGIN_ROOT is empty. Please give a plugin root folder name.")
396        anyError = True
397
398    if not module.PLUGIN_EXE_NAME:
399        print("PLUGIN_EXE_NAME is empty. Please input a name for plugin's exe")
400        anyError = True
401
402    if module.PLUGIN_ICON and not os.path.isfile(module.PLUGIN_ICON) and module.PLUGIN_ICON.endswith(".png"):
403        print("PLUGIN_ICON has a value but the value is invaild. It needs a path to a png file.")
404
405    if module.PLUGIN_EXE_ICON and os.path.isfile(module.PLUGIN_EXE_ICON):
406        if not module.PLUGIN_EXE_ICON.endswith(".ico"):
407            try:
408                import PIL
409            except ModuleNotFoundError:
410                print("PLUGIN_EXE_ICON icon format is not ico and cannot perform auto convert due to missing module. Please install Pillow 'pip install Pillow'")
411    else:
412        if module.PLUGIN_EXE_ICON:
413            print(f"PLUGIN_EXE_ICON is ether empty or invaild path")
414
415    if module.ADDITIONAL_FILES:
416        for file in module.ADDITIONAL_FILES:
417            if not os.path.isfile(file):
418                print("ADDITIONAL_FILES Cannot find", file)
419    
420
421    if not anyError:
422        print("Validation completed successfully, No error found.")
423
424def runBuild():
425	"""
426	runBuild() this takes no arguments. This is the same as `tppbuild file.py` you do not need to pass your build config,
427	It will automatically find it.
428	"""
429	frame = inspect.stack()[1]
430	module = inspect.getmodule(frame[0])
431	file = module.__file__
432
433	main([file])
PLUGIN_MAIN = ' '

REQUIRED PLUGIN_MAIN: This lets tppbuild know where your main python plugin file is located so it will know which file to compile. Note: This can be ether relative or absolute to the main script.

PLUGIN_EXE_NAME = ' '

REQUIRED PLUGIN_EXE_NAME: This defines what you want your plugin executable to be named. tppbuild will also use this for the .tpp file in the format: pluginname + "_v" + version + "_" + os_name + ".tpp"

PLUGIN_EXE_ICON = ' '

OPTIONAL PLUGIN_EXE_ICON: This should be a path to a .ico file. However if png is passed in, it will tries to automatically converted to ico. if PILLOW is installed.

PLUGIN_ENTRY = ' '

REQUIRED PLUGIN_ENTRY: This can be either path to entry.tp or path to a python file that contains infomation about entry. Note if you pass in a entry.tp, tppbuild will automatically validate the json. If you pass in a python file, it will build entry.tp & validate it for you. If validation fails, tppbuild will exit.

PLUGIN_ENTRY_INDENT = 2

OPTIONAL This allows you to set indent for the entry.tp json data. Default is 2 but if you want to save space use -1 meaning no indent. This is only used if PLUGIN_ENTRY is a py file.

PLUGIN_ROOT = ' '

REQUIRED This is the root folder name that will be inside of .tpp

PLUGIN_ICON = ' '

OPTIONAL Path to icon file that is used in entry.tp for category imagepath, if any. If left blank, TP will use a default icon.

OUTPUT_PATH = './'

OPTIONAL This tells tppbuild where you want finished build tpp to be saved at. If leaved empty it will store tpp to current dir where this build config is located.

PLUGIN_VERSION = ' '

OPTIONAL A version string will be used as part of .tpp file. example input PLUGIN_VERSION = "1.0.0-beta1"

ADDITIONAL_FILES = []

OPTIONAL If you have any required file(s) that your plugin needs, put them in this list.

ADDITIONAL_PYINSTALLER_ARGS = []

OPTIONAL Any additional arguments to be passed to Pyinstaller.

def validateBuild()
366def validateBuild():
367    """
368    validateBuild() this takes no arguments. when It's called it will check for all
369    required constants variable are vaild It will check if path is vaild, is a file etc... If 
370    any error is found, It will list all the error during the process.
371    """
372    frame = inspect.stack()[1]
373    module = inspect.getmodule(frame[0])
374    constants = module.__dir__()
375    os.chdir(os.path.split(module.__file__)[0])
376
377    checklist = [attri in constants for attri in attri_list]
378    print("Checking if all constants variable exists")
379    if all(checklist) == False:
380        print(f"{os.path.basename(module.__file__)} is missing these variables: ", " ".join([attri for attri in attri_list if attri not in dir(constants)]))
381        return 0
382
383    print("Checking variable is vaild")
384
385    anyError = False
386
387    if module.PLUGIN_MAIN and not os.path.isfile(module.PLUGIN_MAIN):
388        print(f"PLUGIN_MAIN is ether empty or invalid file path.")
389        anyError = True
390
391    if module.PLUGIN_ENTRY and not os.path.isfile(module.PLUGIN_ENTRY):
392        print(f"PLUGIN_ENTRY is ether empty or invalid file path.")
393        anyError = True
394
395    if not module.PLUGIN_ROOT:
396        print("PLUGIN_ROOT is empty. Please give a plugin root folder name.")
397        anyError = True
398
399    if not module.PLUGIN_EXE_NAME:
400        print("PLUGIN_EXE_NAME is empty. Please input a name for plugin's exe")
401        anyError = True
402
403    if module.PLUGIN_ICON and not os.path.isfile(module.PLUGIN_ICON) and module.PLUGIN_ICON.endswith(".png"):
404        print("PLUGIN_ICON has a value but the value is invaild. It needs a path to a png file.")
405
406    if module.PLUGIN_EXE_ICON and os.path.isfile(module.PLUGIN_EXE_ICON):
407        if not module.PLUGIN_EXE_ICON.endswith(".ico"):
408            try:
409                import PIL
410            except ModuleNotFoundError:
411                print("PLUGIN_EXE_ICON icon format is not ico and cannot perform auto convert due to missing module. Please install Pillow 'pip install Pillow'")
412    else:
413        if module.PLUGIN_EXE_ICON:
414            print(f"PLUGIN_EXE_ICON is ether empty or invaild path")
415
416    if module.ADDITIONAL_FILES:
417        for file in module.ADDITIONAL_FILES:
418            if not os.path.isfile(file):
419                print("ADDITIONAL_FILES Cannot find", file)
420    
421
422    if not anyError:
423        print("Validation completed successfully, No error found.")

validateBuild() this takes no arguments. when It's called it will check for all required constants variable are vaild It will check if path is vaild, is a file etc... If any error is found, It will list all the error during the process.

def runBuild()
425def runBuild():
426	"""
427	runBuild() this takes no arguments. This is the same as `tppbuild file.py` you do not need to pass your build config,
428	It will automatically find it.
429	"""
430	frame = inspect.stack()[1]
431	module = inspect.getmodule(frame[0])
432	file = module.__file__
433
434	main([file])

runBuild() this takes no arguments. This is the same as tppbuild file.py you do not need to pass your build config, It will automatically find it.