So Daz Content Manager detects what packages are in the Daz Install Manager Downloads folder (where you keep all the packages). If you drag the non-DIM into the corresponding asset in Content Manager, it gets packaged up and placed in the Download folder. Then you use Install Manager to install the package. It even creates a Smart Content product, but it's placed in Lost and Found and you have to move it to the category you want.
OK.
I fixed the issue with memory problems, at least, partly.
The problem is that by default, Java installs as 32 bit. This application requires a 64-bit Java environment to work properly.
This might be a good idea to mention in the OP.
I fixed the issue with memory problems, at least, partly.
The problem is that by default, Java installs as 32 bit. This application requires a 64-bit Java environment to work properly.
This might be a good idea to mention in the OP.
- Sep 22, 2017
- 675
- 800
Not sure what to do as the video is no help, the is only a start item and a .jcs or something but window's will not allow to run. If you bypass window's you get a white screen which goes see throw then it disappears. not sure what happened but don't this it went well. Help if you don't mind, thanks.
If it was possible, I'd like to migrate this to something a bit easier to maintain and develop, like Python with Django...
But I would need other developers for this as my knowledge is not enough.
But I would need other developers for this as my knowledge is not enough.
Is there any way to do something about the browser user agent the software passes?
Because 3d-load.net blocks access as soon as you have scraped 25 pages. They did for me, at least.
But I can load the site fine with Edge and Chrome.
Alternatively, does anybody have their full downloadpages scraped?
I will upload the downloaded asset library shortly. Luckily, I am on a gigabit connection.
Because 3d-load.net blocks access as soon as you have scraped 25 pages. They did for me, at least.
But I can load the site fine with Edge and Chrome.
Alternatively, does anybody have their full downloadpages scraped?
I will upload the downloaded asset library shortly. Luckily, I am on a gigabit connection.
For the moderators:
Here is a current folder with scraped assets:
Here is a current folder with scraped assets:
You must be registered to see the links
- May 14, 2019
- 379
- 755
Curious if rythamun had to abandon this program?
It has been my go to, #1 Daz asset for months now.
It looks like 3D-LOAD.net has implemented a captia system, or made some changes so that its newest assets are not showing up in the DCM downloads pane.
Ideas?
(or, is anyone else having problems scraping 3D-Load.net?)
It has been my go to, #1 Daz asset for months now.
It looks like 3D-LOAD.net has implemented a captia system, or made some changes so that its newest assets are not showing up in the DCM downloads pane.
Ideas?
(or, is anyone else having problems scraping 3D-Load.net?)
Last edited:
I am having problem with 3D-LOAD.net as well.
For me it seems the site is currently protected by cloudflare and the sw can't handle that.
Would be great it that could be fixed since 3D-LOAD is the "easiest" scraper with the best free file hosters.
- Jun 19, 2020
- 57
- 30
This thing is precious as idea... But I'm having a hard time to make it work consistently -- sometimes it doesn't show images, scraper empty for a long time (
Edit: I see some "heap space" errors happening too when importing the assets. Sounds like a memory issue, any way to set more memory to Java?
EDIT 2: as pointed by someone in this topic, for the version 0.5 only Java x64 do the work. Download it here
Now it's working as intended with a minimal issue: it doesn't remove the installation files even if we turn the option. I have to do it manually to don't eat double hdd space. Amazing tool anyway.
Aa a suggestion if development continues, I would love to see Render State dot Net site among the scrapers. They have files in Google Drive, that is fast as hell and with no captcha (not really sure if it's possible).
You must be registered to see the links
), etc... If I leave "Run at startup" turned On for all vendors, do it try to re-scrape everything all times?Edit: I see some "heap space" errors happening too when importing the assets. Sounds like a memory issue, any way to set more memory to Java?
EDIT 2: as pointed by someone in this topic, for the version 0.5 only Java x64 do the work. Download it here
You must be registered to see the links
.Now it's working as intended with a minimal issue: it doesn't remove the installation files even if we turn the option. I have to do it manually to don't eat double hdd space. Amazing tool anyway.
Aa a suggestion if development continues, I would love to see Render State dot Net site among the scrapers. They have files in Google Drive, that is fast as hell and with no captcha (not really sure if it's possible).
Last edited:
- Aug 12, 2019
- 83
- 205
New renderosity items don't work. The file names of the images are longer than 255 characters.
With which free database app you can open the db file. Have already tried a few but they don't recognize the db file.
Problem solved.:
I have open the db file with notepad++ en change :
"MEDIA_SRC" VARCHAR(255) to "MEDIA_SRC" VARCHAR(800) (32 times)
With which free database app you can open the db file. Have already tried a few but they don't recognize the db file.
Problem solved.:
I have open the db file with notepad++ en change :
"MEDIA_SRC" VARCHAR(255) to "MEDIA_SRC" VARCHAR(800) (32 times)
Last edited:
Sadly, I don't have time for this project right now and I don't even know when or if I will continue the development at all.
But if someone wants to continue or at least fix the problems. Here is the source code:
I created the project because doing things manually annoyed me - but I never intended to share the project.
So the code should be ok-ish, but quick and dirty at times - and of course without any documentation at all
If you are familiar with Java/Spring and Angular, you should be fine...
But if someone wants to continue or at least fix the problems. Here is the source code:
You must be registered to see the links
I created the project because doing things manually annoyed me - but I never intended to share the project.
So the code should be ok-ish, but quick and dirty at times - and of course without any documentation at all
If you are familiar with Java/Spring and Angular, you should be fine...
Just want to say I really appreciate you sharing it with us. Has made my life much easier, especially when I buy non-DAZ-approved assets and need to get them installed properly without a total mess in my filesystem.Sadly, I don't have time for this project right now and I don't even know when or if I will continue the development at all.
But if someone wants to continue or at least fix the problems. Here is the source code:
You must be registered to see the links
I created the project because doing things manually annoyed me - but I never intended to share the project.
So the code should be ok-ish, but quick and dirty at times - and of course without any documentation at all
If you are familiar with Java/Spring and Angular, you should be fine...
- May 14, 2019
- 379
- 755
I have two db files...mv and trace. Do you have to do it for both? If not which one?
Seriously, if you decided to support this again, I'll be your first patron.Sadly, I don't have time for this project right now and I don't even know when or if I will continue the development at all.
But if someone wants to continue or at least fix the problems. Here is the source code:
You must be registered to see the links
I created the project because doing things manually annoyed me - but I never intended to share the project.
So the code should be ok-ish, but quick and dirty at times - and of course without any documentation at all
If you are familiar with Java/Spring and Angular, you should be fine...
- Aug 12, 2019
- 83
- 205
- Jun 19, 2020
- 57
- 30
I really hope some Java developer get it and keep it going. It's fantastic. Thanks for sharing anyway.
EDIT: besides, I just noticed Renderosity is blocking by IP after we start to scrape. Soon as the process started, I wasn't able to access it on browser anymore, than using a proxy, everything fine. Bad...
Last edited:
- May 14, 2019
- 379
- 755
I tried using realteamkiller's trick above (switching from 255 to 800) but that didn't work for me (plus, it was a bear to find a program that would open my .db file that is over 2GB).
Curious if anyone is interested in taking up the challenge of keeping this awesome program going. I downloaded the source files, but it is outta my league.
Curious if anyone is interested in taking up the challenge of keeping this awesome program going. I downloaded the source files, but it is outta my league.
- Aug 12, 2019
- 83
- 205
Strange that it works for me.
Because this program no longer has support, I'm just making my own access db. I am using octoparse (free app) to scrape the data.
So standard tools.
Last edited: