Tool Others Vambropy [v1.18]: VaM packages & user content database management, and a complement to VarBrowser

commondi32

Newbie
Jul 16, 2020
84
62
29
1764384667270.png

Overview
Vambropy is an open source tool for managing VaM package collections and user content. It is based on sfishere's VarBrowser ways to help VaM performance when the collection grows, by moving files within the same VaM directory, between AddonPackages and AllPackages directories (more efficient and robust than symlinking/renaming single files to enable/disable).
It uses a small database file to import relevant data from VaM packages and user content to allow easy search, filtering, rating, and to create/update custom lists which can dynamically activate/deactivate custom set of packages at once, enhancing VaM performance and responsiveness for different use cases. Thus, creators can configure more strategic profiles based on tools needed for their creations, and users can fast-switch and optimize environments like VamX, VamProduce 69, or any other desired configuration
Integration with VarBrowser (not needed but recommended) is provided by using same structure, with enhanced control over dependencies too, allowing to selectively install/ignore as well. Collections on external drives can be added to the database keep track of every .var file's info even when drives are offline. Synchronization with Varbrowser's AutoInstall tagging system provides a faster and more flexible way than the plugin itself (setting the tag on multiple packages at once). Sorting and filtering option makes it easy to find morph-heavy packages, very dependent ones, very depended ones, etc.

This update has numerous enhancements, changes and fixes. Some less noticeable, some bold ones.

Features :
  • A high-performance sqlite database
  • VarBrowser's concept of available/installed packages for better VaM performance.
  • VarBrowser's Autoinstall tag system enhanced (toggle on/off selection of packages at once)
  • Packages and related dependencies sections with context menu options and hotkeys.
  • Easily see and install sub-dependencies VarBrowser left uninstalled.
  • Easy package installation (copy) from external package repositories (other drives/dirs)
  • Older/stale packages filtering and moving options.
  • Ignore option for specific dependencies.
  • Forced install of latest versions option.
  • Simple but effective rating system, practical to remember good stuff. :BootyTime:NEW
  • Deep filtering, find dependencies, morphs, description, clothing and hair single items NEW.
  • NO DELETION/EDITION SO NO PACKAGE CAN BE ALTERED OR LOST.
  • Packages are handled preserving their relative paths as they are, everywhere.
  • User content browsing with images and basic info, dependencies, categories filtering.
  • Prevents duplicates in VaM and warns about bad /corrupted/incomplete packages.
  • Pointer-drag selection with file size sum info on title bar. (Shift+Ctrl modifiers support)
  • UI Themes, each with configurable color-coded info for package and deps rows. NEW
  • All package's thumbnails in a single image, and single images viewable in each package (image scroll). NEW
  • Custom lists (profiles) allow to install/uninstall saved groups of packages at once. UPDATED
  • Export selections to clipboard. (Easily copy/paste missing deps in Sharp VaM Tools)
  • Find and download free dependencies directly from Hub. NEW
  • Highlighted packages containing VarBrowser's favorite items.
  • Highlighted older packages/dependencies.
  • Image detaching option separates the image from main UI, resizable.
  • Dependency usage for each package (related to the whole database collection).
  • Sym-Linked package install from Repos (external NTFS partition/disks). (*1)
  • Unpacking option for clothing and hair. (*2)
(*1) File symlinking was a requested feature that only works when running the app with Administrator privileges. Generally not recommended (use manually created folders symlinks instead).
(*2) Unpacking might be useful mainly for clothing, it can be slightly faster than vars when loading person atoms (with some compromises, not recommended*).

Powerful multiple group and multi-word filters:
Browse specific content by package name, creator, tags, description (useful for plugins), morphs items, clothing and hair items, creation date, with an instant, as-you-filter related dependencies panel which is also filterable by name and availability status).
Example: just type 'camera' in description filter and all related packages will show; type 'scripts' or '*scripts' in tags to reduce even more, refining the search for that camera-related plugin.)

Special folders NEW:
A planned feature is file organizing, aiming to help at file level, letting you spend less time manually moving files with Explorer or any OS tools. Like feelfar's Var Manager did, but not automated and configurable, with more options. Currently, there are 4 new folders to move packages, configurable:
  • _newvars is where hub downloads are saved. Default is AddonPackages\_newvars. Could be something like AllPackages\_newvars if you prefer new downloads to be available but not yet installed.
  • _oldvars is a 'Move to...' menu option, where you can move old/stale vars. Default is _oldvars (root is VaM; it could be also set to AllPackages\_oldvars if you want to let the old versions still be available)
  • _removed is a 'Move to...' menu option, where you can move unwanted packages. You can later review them and manually delete, backup, etc.
  • _favorites is a recently requested 'Move to...' menu option, where you can send your favorite packages. Default is AddonPackages\_favorites. At the moment, when it is set to installed zone (AddonPackages), installed (including autoinstall), and not installed packages are moved, so uninstalled ones will be installed. As always, all the sub-paths are kept. (*)
All 'Move to...' menu options are protected against nested folders of the same type (i.e. ...\_removed\_removed\_removed\...), so it is no problem is you accidentaly move the already moved package again. They will be ignored.

(*) Any of these folders could be replaced with symlinked folders (do it at your own risk - there is no problem at all moving files to symlinked folders in another drive, but it was not tested enough). The _favorite folder idea was originally thought as a symlinked folder. Example: to help in a situation where a new faster SSD is bought and you want to start moving your favorite packages to it, so you can replace your _favorites folder with a symlink to that SSD, avoiding to browse your var files through OS app, unzipping vars to see what's on it. Just use Vambropy to browse, select and move. Feedback on it is very welcome.


IMPORTANT (FAQ):

Q: Does it modify my AddonPackages folders structure, vars , content?
A: Vambropy does NOT modify and currently is not aiming to modify* any files other than Custom/PluginData/sfishere/Autoinstall.txt (and Favorites,txt). It does NOT modify your sub-directories structures. When you command it to install/uninstall, it moves with their paths between AllPackages and AddonPackages dirs, so all paths are kept intact; same with installing by copying from another drive and directory. You can configure your external drives/ directory trees according to your own structures and the way it copies from external sources into VaM.

*There are plans, however, to link custom tools to it, giving you as much more power as you need, without being an all-around app which does everything (since it increases code complexity -> harder to maintain -> more time-consuming -> abandon)
Custom tools could be anything that can batch-process selections, for any tasks (i.e. modifying /debloat, etc.) but always externally, independently. This would be simple, like exporting the selection as a list of files with full path to be picked by the tool for processing/copying the selection to paste in the tool. So anyone could integrate any tool with Vambropy. Only requirement for the external tool is a command line option to accept a list of paths/a drop zone to throw the list/paste from clipboard, like Sharp Vam Tools does. So, just code the tool/ask for path list support to existing tools if they don't. Simple, isn't it? :WeSmart:)

Q: It is a VaM plugin? Does it need VaM to be running along?
A: Vambropy is independent of VaM.exe, it does not need VaM/VarBrowser to be running, and changes made with it are not automatically reflected in VaM. if you are running VaM and Vambropy together, after making changes on one of them you will need to refresh the other. (A better way of refreshing packages in VaM -to avoid its low-performant package manager- is binding a hotkey through Keybindings plugin by AcidBubbles.)

Q: You said it does not delete any .var file but there is a dialog at start about removing duplicates. :WaitWhat:
A: When Vambropy starts, it checks for duplicates between AllPackages/AddonPackages, giving the user the option in a dialog box to keep just one. The rest is deleted ONLY IF they are digitally the same file, and it cannot accidentally delete all of them, always keeping the one you decide to keep. Plus, if they are not digitally equal, the duplicates are moved out and renamed (a 'DUPE_NOT_EXACT.' prefix is added) to allow you to review it later.



About the code:
Vambropy is python-coded, open source; code is fully commented on what it does and is easily moddable if you know some programming basics. qube21 is also working a lot to enhance functionality and interface, with numerous great features (see Vambropy.pyw changelogs). If you are are a python coder and want to mod it/fix things/enhance functionality just jump aboard! If you want to make it bigger or part of another project just mention me as original creator and the current collaborators as well.



Version 1.17 is kept downloadable, mainly for Qt version (not available at the moment in new version).

Qt version by qube21 features a high-performance UI with native high DPI support.
vambropy-qt-Window.png



Some details on its workflow (still needs some updating):

You don't have permission to view the spoiler content. Log in or register now.

You don't have permission to view the spoiler content. Log in or register now.

You don't have permission to view the spoiler content. Log in or register now.

You don't have permission to view the spoiler content. Log in or register now.

You don't have permission to view the spoiler content. Log in or register now.



IMPORTANT FOR v1.18: Before you start it, you can use the same thumbnails folder from 1.17, but vambropy.db can't be used, it has to be created again. If you have set some ignore tags on dependencies, you can export them and import after new database is created. Use VBDB tool for that. Copy the EXE or .py file in Vambropy folder. Type 'vbdb.exe' or 'python vbdb.py' export' in the command line and the the usage options. 'vbdb export' will export a list in a text file for those dependencies. Then make sure that text file is there along with the new vambropy.db, and call the tool again, this time 'vbdb import'.
The tool also exports/import the new user-set package ratings, in case of future db rebuild.


FOR v1.17 ONLY
: In the binary download there are 2 executables: vambropy and vambropy_qt. If you use the User content section, use first the standard vambropy.exe to switch to the user content section, scan the content and update the database (the qt version has a bug at the moment, not finding the jsons). After that, you can close the standard version and open vambropy_qt. It will ask again for vam directory (different config files) but it uses the same database.


ABOUT VIRUS WARNINGS: The binary release has been modified by leaving resources out of the executable, to prevent false VIRUS positives. However, as a pyinstaller executable/uncertified windows app it may still raise alerts in antiviruses. Please be aware of this. If you are unsure about it, you can check the code and execute it from its sources with python 3.12, even build your own .exe( read the Readme.txt file on how to do it and/or check out the thread for help already offered on that).
The sources include 2 .exe files, GNU grep and sed (in resources dir), which might raise VIRUS alert here. You can also them from original sites.

Download
v1.18 binaries (Tk)
V1.18 Tk sources (see attachment)

Older versions:
v1.17.3 binaries Tk-only (fixes UI bugs)

V1.17 binaries (Qt version here)
 
Last edited:

Trebutor

New Member
Mar 23, 2020
1
0
11
Nice, but somethings are missing:

- Dependencies, I assume Tk and Image.
- How to run, I assume "python vambropy.pyw"
- What to put in config.ini?
- It doesn't seem to do anything right now. Is there a magic hidden button to press?
 

commondi32

Newbie
Jul 16, 2020
84
62
29
Nice, but somethings are missing:
...
Thanks for the feedback. Yes, maybe you need pillow (pip install pillow), for images. Tkinter comes bundled with python 3.11 I think.
I have been testing and fixing bugs in the Dependencies section (like buttons not working yet) among other things. I have been testing as many as I can, and now uploading new version with many changes and fixes, some optimizations, and update on all help about new config.ini (reworked with more settings and comments). I think it is now easier and more solid and interactive, with more messages, warnings to avoid little accidents, and customization options. It should run right from the unzipped files. To run by src, go to src dir and 'python vambrpoy.pyw' should work. Or just open vambropy.exe (no python needed). Start the program, click the "Edit config.ini" button and set the correct VAM_dir. Then restart.
 
Last edited:

commondi32

Newbie
Jul 16, 2020
84
62
29
I have not much experience uploading and attaching files, and today I managed to upload the new version with executable + sources in the same file because it is smaller now and it fitted. But the legend [VIRUS] was inserted in the attachment name. What is that? Do I need to upload it again? I am sure the file is not infected, I cannot tell why that happened. If there is problem to download that way please tell me and I'll upload separated binary from source if needed.
 

commondi32

Newbie
Jul 16, 2020
84
62
29
Just a suggestion, but why not just use symbolic links instead of moving the files?
It could be an option. I was not thinking about it because I don't use it personally, and the app was first conceived to complement sfishere's VarBrowser. And there is already feelfar's VarManager doing symlinks, and recently BrowserAssist. Also, I thought maybe people do not want to try a unknown tool that is asking for admin privileges, since symlinks cannot work at user level. I'll be releasing an update 1.08 soon to fix some bugs introduced in later versions, and then I will try to do some tests. If I manage to make it work and optionally ask for admin when it enables it, I'll add it.
 
Last edited:
  • Like
Reactions: Franknar

commondi32

Newbie
Jul 16, 2020
84
62
29
Getting this error once all packages are loaded by the program, any idea how to fix?: View attachment 4581308
Thanks for the feedback. I think this was one of the bugs fixed in the new version 1.08
EDIT: If it keeps showing on the new version, the problem seem to be with a dependency in a .var file that requires a minimum version, so it is expecting a number but it gets a 'latest', I imagine an incorrect string like "minlatest", as if it was a manual correction to a meta.json, since some people mess with those files manually and corrupt the file. Could be a little hard to pinpoint. If you have rare vars not from the hub or known sources, you can try inspect those vars with VaM Sharp Tools, it might find and fix some bad stuff.
Another way (you may not like much) is to rename your current packages.db and start again with a limited list of vars. If you previously include repos, do it only with vam directory this time, see if it starts without problems, and then add bunchs of vars to the database until the exception shows again
EDIT AGAIN: I will try to add some code to manage that kind of execption in the next version, so it can skip the (bad) package in case of a malformed string , not throwing you out of the app.
 
Last edited:

commondi32

Newbie
Jul 16, 2020
84
62
29
Just a suggestion, but why not just use symbolic links instead of moving the files?
Well it turned to be somewhat easy to implement it as an alternative copy method. The menu now it has the option to symlink your package(s) selection, or by pressing <L> key. It works on packages windows, at the moment. Check info on the release (symlink spoilers section) for details. Remember to run Vambropy as admin. If not, options are available but will not do anything more than create relative subdirectories for the links.
Symlinks will not mix with packages, they will copy in AddonPackages/symlinked. And if uninstalled, they will go to AllPackges/symlinked, preserving relative paths.
 
  • Like
Reactions: Franknar

Franknar

New Member
May 12, 2017
7
2
154
Well it turned to be somewhat easy to implement it as an alternative copy method. The menu now it has the option to symlink your package(s) selection, or by pressing <L> key. It works on packages windows, at the moment. Check info on the release (symlink spoilers section) for details. Remember to run Vambropy as admin. If not, options are available but will not do anything more than create relative subdirectories for the links.
Symlinks will not mix with packages, they will copy in AddonPackages/symlinked. And if uninstalled, they will go to AllPackges/symlinked, preserving relative paths.
Amazing! Thanks
 

yuezy3

New Member
Mar 29, 2025
11
5
3
great work, but this config.ini may cz program fail because in line 1006 of modules/ui.py :

with open(config_file_path, 'r') as file:

read config.ini but not sepcify encoding. change this line to :

with open(config_file_path, 'r', encoding='utf-8') as file:
will better. it will now can handle other non assic char as path.
 

cdt_drecks

New Member
Dec 7, 2024
7
1
47
Hi, I get this error:

Traceback (most recent call last):
File "vambropy.pyw", line 79, in <module>
File "vambropy.pyw", line 68, in main
File "modules\ui.py", line 1184, in refresh
TypeError: 'NoneType' object is not subscriptable
 

commondi32

Newbie
Jul 16, 2020
84
62
29
Hi, I get this error:

Traceback (most recent call last):
File "vambropy.pyw", line 79, in <module>
File "vambropy.pyw", line 68, in main
File "modules\ui.py", line 1184, in refresh
TypeError: 'NoneType' object is not subscriptable
Hi, you have found a bug in favorites.txt file handling. The program should create an empty one if it does not exists, but somehow an exception occurs. I will fix it in next release. In the meantime, you can fix it two ways:

Simple way: In VaM, use Varbrowser and set any package you like as favorite (it turns blue). That will create the needed file; or

Edit way: Create an empty "favorites.txt" in Custom/PluginData/sfishere directory (the path is inside VaM directory). Open the empty file with a text editor and type: {"FavoriteNames":[]}. Save the file.

Then Vambropy will start ok.
 
  • Like
Reactions: cdt_drecks

cdt_drecks

New Member
Dec 7, 2024
7
1
47
Hi, you have found a bug in favorites.txt file handling. The program should create an empty one if it does not exists, but somehow an exception occurs. I will fix it in next release. In the meantime, you can fix it two ways:

Simple way: In VaM, use Varbrowser and set any package you like as favorite (it turns blue). That will create the needed file; or

Edit way: Create an empty "favorites.txt" in Custom/PluginData/sfishere directory (the path is inside VaM directory). Open the empty file with a text editor and type: {"FavoriteNames":[]}. Save the file.

Then Vambropy will start ok.
Thansk! Now I have this error:


Traceback (most recent call last):
File "vambropy.pyw", line 85, in <module>
File "vambropy.pyw", line 74, in main
File "modules\ui.py", line 1224, in refresh
File "modules\packages.py", line 66, in find_var_names
File "modules\packages.py", line 47, in find_in_all_packages
UnboundLocalError: cannot access local variable 'latest_key' where it is not associated with a value
 

commondi32

Newbie
Jul 16, 2020
84
62
29
Thansk! Now I have this error:


Traceback (most recent call last):
File "vambropy.pyw", line 85, in <module>
File "vambropy.pyw", line 74, in main
File "modules\ui.py", line 1224, in refresh
File "modules\packages.py", line 66, in find_var_names
File "modules\packages.py", line 47, in find_in_all_packages
UnboundLocalError: cannot access local variable 'latest_key' where it is not associated with a value
Another one, thanks! Fixed in new version 1.10!
 
  • Like
Reactions: cdt_drecks

cdt_drecks

New Member
Dec 7, 2024
7
1
47
Another one, thanks! Fixed in new version 1.10!
Hey, this error is still present with 1.10 :

Traceback (most recent call last):
File "vambropy.pyw", line 89, in <module>
File "vambropy.pyw", line 78, in main
File "modules\ui.py", line 1224, in refresh
File "modules\packages.py", line 70, in find_var_names
File "modules\packages.py", line 51, in find_in_all_packages
UnboundLocalError: cannot access local variable 'latest_key' where it is not associated with a value
 

doggava

Newbie
Dec 16, 2021
82
32
109
This tool looks cool!

By the way, check this tool out too for development ideas:


It e.g. removes old versions or vars and has a mapping functionality for missing vars (using symlinks).
 

luna.kitty

Newbie
Jun 6, 2020
29
15
126
" In future relases a button could be included to do either: a) in Dependencies section, install all (or filtered) external dependencies as symlinks, or b) Install everyhing external symlinked, as 'Available' , so Varbrowser could handle everything from VaM. "

I really need this useful function, because my tons of var files are separated across several storage locations. I want to install dependencies as symlinks in just two steps - click a specific var, then install dependencies from a Dependencies list as symlinks.
I think this tool is the best VAM tool for me, thank you very much!