Advertisement

CHANGELOG

What's New in version 1.8.1.725:

Added:

  • Link Checker GUI client now allows the checking of external links to be enabled or disabled
  • Link Checker GUI client now allows if URLs belonging to parent, sibling or sub domains should be checked
  • Added auto scroll option to Link Checker GUI client
  • Added progress indicator to Link Checker GUI client
  • Added new Use Recycle Bin option to project settings. If set and the Empty website folder before copy is also set, any deleted files will be moved to the Recycle Bin instead
  • The View Links dialog now allows the display of excluded URLs to be toggled
  • Added proper editor for defining web page language settings at the project level
  • Add application level setting for definition web page language settings
  • List exports now present a configuration dialogue for which columns to include the export [#275]

Changed:

  • WebCopy will now prompt to continue if the Empty website folder before copy option is set and files are present in the destination
  • The Sitemap Extension will now start from the base domain if the project URL is deep and the Crawl Above Root flag is set
  • Updated mime-db to 1.44.0
  • The GUI now displays a proper progress indicator and status information when remapping local files
  • The CLI client now displays status information when remapping files
  • The Origin Report option for new projects now defaults to Single File rather than Embedded
  • WebCopy will now always send the Accept-Language header. If not defined at the project level, it will use the application level setting. If this is not provided, then the current OS culture information will be used
  • Documentation has had a good overhaul and is in the best state it has ever been in. All help links from option dialogue boxes point where they should, and missing documentation has been added
  • Expanded default contentfilters.json used by the New Project Wizard to cover other common types
  • The Accepted Content Types field has been moved from the Advanced category into a category of its own, expanded to use the same type of editor as for the web site language

Fixed:

  • WebCopy was treating any attribute value that started with javascript as unsupported
  • The sitemap tree could display duplicate URLs
  • The sitemap tree would could incorrectly display children of pages that matched a standard document pattern
  • Link Checker didn't follow internal redirects
  • WebCopy could incorrectly parse the URL from an @import at-rule if the CSS was minified and another rule contained an empty content declaration
  • The Project Diagnostics extension now ignores data URLs when performing length checks
  • Cut, Copy and Paste commands didn't work for the filter fields in list views
  • Fixed a crash that could occur when ordering the sitemap
  • Reworked HEAD support detection to be more robust
  • 401 challenges were only processed during HEAD requests
  • Fixed a performance issue running XPath queries
  • Per-URL origin reports could be overwritten if URLs differed only by extension
  • The New Project Wizard no longer creates duplicate rules if content types are present in multiple pre-defined groups
  • Fixed a crash that could occur when closing the options dialog after switching views
  • Windows that save their position and size should no longer keep increasing in size each time the window is opened and a custom font is being used with a point size above 8
  • Options dialogues are now slightly more usable when using custom fonts with a point size above 8
  • The Quick Scan dialogue now correctly disables the Scan button when busy, preventing a crash trying to perform multiple scans
  • Setting the URL in the main window now correctly defaults http if a scheme is not explicitly set, preventing a crash when using secondary actions such as trying to capture a form
  • The New Project Wizard dialogue now ensures that user entered URLs have a default scheme applied if omitted by the user
  • WebCopy could incorrectly parse blank url CSS functions
  • Fixed inconsistencies in when the Download All Resources option would be enabled or disabled
  • Fixed a crash posting a blank form definition

Removed:

  • The Report Problem Site extension is no longer bundled with Setup
  • Removed global statistics

What's New in version 1.8.0.652:

  • Warning! WebCopy projects saved using 1.8 are not compatible with older versions. If you use a nightly/beta version of 1.8 we strongly recommend you keep a backup of any project files saved using this version in case you wish to revert to an older version
  • Fixed a crash which occurred if contentfilters.json was not present in &'37;appdata&'37;CyotekWebCopy1.0

What's New in version 1.8.0.651:

Added:

  • Added additional options to proxy server configuration, allowing the use of system proxies and user defined bypass lists
  • The poster attribute of video elements is now detected

Fixed:

  • Fixed a crash that could occur when verifying the initial path
  • The Internet Explorer DOM provider failed to process some pages if an attribute request failed with an DISP_E_TYPEMISMATCH error
  • The Limit Crawl Depth, Limit Distance from Root URL, Maximum Files, Maximum File Size and Minimum File Size options would be processed if they had previously been set even if subsequently disabled
  • Proxy settings dialog now does a better job of validating the address
  • Fixed a crash when clicking help links in stand alone tools
  • Fixed a crash which could occur when using the Select URI dialog
  • Speculative fix for a crash when painting list views
  • Speculative fix for a crash trying to capture a form
  • Speculative fix for a crash setting the same folder

What's New in version 1.7.0.600:

Changed:

  • URI properties dialog are now opened as non-modal windows if possible

Fixed:

  • Quick Scan dialog temporary restricts maximum number of displayed pages to 200, resolving a crash that occur on sites with thousands of detected pages
  • Fixed an issue where WebCopy would always full download files above the copy root if the Download All Resources option was enabled
  • Fixed an issue where WebCopy wouldn't correctly exclude entries above the root if the Download All Resources option was enabled
  • Files over 2GB in size wouldn't be downloaded
  • After 2147483647 files had been downloaded, no further downloads would occur
  • Setup programs were only signed with SHA256, meaning Windows Vista couldn't read the signatures
  • Setup tried to install .NET 4.6.2, causing an installation failure on Windows Vista which only supports 4.6.0

What's New in version 1.7.0.583 Beta:

Added:

  • It is now possible to authenticate with a website using an embedded web browser prior to copying, allowing WebCopy to work with sites that have complex login procedures or multi-factor authentication [#333]
  • When copying a website with an SSL certificate, if the Ignore certificate errors option is not set, WebCopy will now display a dialog asking what to do [#329]
  • Added a new option to include the original extension when remapping files [#324]
  • Added a new option to include the query string in local filenames [#267]
  • Added new options for limiting downloads based on file size (minimum and maximum) and on the number of files downloaded
  • Added Last Modified column to various URL list views
  • Added URL browser dialogs to various selection fields

Changed:

  • Reinstated URI editor for URI Transforms
  • URL browser dialogs now keep the original selection

Fixed:

  • HTML entities in attribute values were not decoded when scanning for links
  • Local files no longer have their extensions changed if the URI extension doesn't match the first extension in the content type database, e.g. .jpg files no longer get renamed to .jpeg
  • Sitemap generation now correctly ignores redirected URL's
  • WebCopy could incorrectly abort a download with an insufficient disk space error if even free space was available

What's New in version 1.6.0.559:

  • No progress dialog was displayed when clicking the Copy or Scan buttons to the right of the URL field

What's New in version 1.6.0.555:

Added:

  • Added new nocrashreport switch to command line clients
  • Command line clients can now display solution information when reporting crashes [#201]

Fixed:

  • Partial output is no longer printed by CLI tools when using the quiet switch
  • Statistics are now printed when using the statistics switch even if quiet is also specified
  • All output is now correctly written to log files when the log switch is used, irrespective of the quiet switch setting
  • Pressing Enter or Escape in the Capture Form dialog no longer closes the dialog if the embedded web browser has focus

What's New in version 1.6.0.551 Beta:

  • The progress dialog was no longer being displayed [regression]

What's New in version 1.6.0.543 Beta:

New:

  • The Basic Authentication dialog now allows the prompting of future passwords to be disabled
  • Preview functionality of the Test URI dialog now supports a subset of images
  • Added proxy settings to Test URI dialog
  • Added new options for specifying custom headers [#219]
  • The Test URI dialog now allows the configuration of content encoding, custom headers and URI transforms [#296]
  • Added stand-alone version of the test URI tool
  • The Capture Form dialog will now try and find the best match if multiple forms are detected on a page [#230]

Changed:

  • The layout of the Test URI dialog has been reworked [#296]
  • Setup has a new option to determine if icons should be created for stand-alone tools
  • Setup has a new option to determine if experimental 64bit versions of tools should be installed
  • Minor improvements to External Tools dialog
  • Minor start-up improvements
  • The option to save headers with the project file is now enabled by default for new projects
  • Some context menu items which disappeared from virtualised lists have now been re-instated
  • If the character set for a HTML document isn't explicitly specified, WebCopy will now try and autodetect an appropriate value [#303]

Removed:

  • Removed the Content tab from the Link Properties dialog
  • Removed the unused Modified URI field from the Link Properties dialog
  • Removed Find more user agents online link from the Edit User Agents dialog
  • Removed the Allow Editing checkbox from the Link Properties dialog
  • Removed the Disable Updates flag from link information

Fixed:

  • The Basic Authentication dialog truncated long realm text [#312]
  • WebCopy no longer tries to unpack custom settings belonging to unloaded extensions [#278]
  • WebCopy no longer stores downloaded content against the link information when a 400 or 500 series response is returned
  • Output editors in the Test URI dialog now honour the Fixed Font setting
  • Project files were no longer being compressed when saved
  • Fixed a crash that could occur when running the Empty Meta Data report
  • Backup files were not being created when saving projects
  • Default external tools configurations were not added when starting WebCopy for the first time
  • Some files were still download even if they had been excluded via a rule (regression from 1.4)
  • Editing a local file using the build in text editor always used UTF-8 and would corrupt files using a different encoding
  • The default user agent was using the file version of the WebCopy client instead of the product version
  • The Quick Scan window is now resizeable and remembers its position
  • Corrected some settings that weren't being cached

What's New in version 1.5.0.516:

Added:

  • Added a new diagnosis extension to help investigate certain project errors which are not reproducible in current test data
  • Added new exclusion options to more finely control the remap extension mode
  • The Content-Disposition header is now supported and if set will help define the local filename

Changed:

  • Tabbed or tree based option/property dialogs now include a search field
  • Split the Copy options page into Folder and Local Files pages

Fixed:

  • Uninstall should no longer prompt for feedback when running Setup to upgrade an existing installation
  • Fixed an issue where the Download all resources setting was switched off when opening the options dialog (regression)
  • Fixed a crash which could occur when clicking the Test URI button in a form editor for a project with no base URI set
  • Speculative fix for a crash which could occur when deleting an empty rule or form
  • Speculative fix for a crash which could occur when displaying the Select URI dialog
  • Setup was installing the Problem Site Report extension into the wrong folder, overwriting the RSS extension manifest
  • Strings over 32767 bytes in size are now supported in WebCopy projects
  • Pressing Enter in multi-line edit fields in the Inclusions / Exclusions option page closed the dialog
  • Fixed a number of cases where modifying a collection might not mark the project as changed
  • Fixed a crash that could occur if WebCopy couldn't get an encoding [#304]

What's New in version 1.4.0.469:

  • Fixed: The improved Quick Scan dialog crashed if the Visual Link Map extension wasn't installed
  • Fixed: If the URI to crawl redirected to an external URI, no feedback would be provided to the user regarding the redirect and the crawl would just appear to halt with an empty response
  • Fixed: A crash no longer occurs if a website returns a Content-Encoding header that either isn't a standard value or one that is not supported by WebCopy
  • Fixed: List views now remove line breaks from displayed content

What's New in version 1.4.0.465 Beta:

Added:

  • A Rule Checker tool has been added, which takes a given URI and passes it through all rules, allowing you to see which rules are matched and which aren't
  • Added new Stop processing more rules flag. This flag is automatically applied to projects created using older versions of WebCopy
  • List filters previously removed as part of [#65] have now been reinstated
  • List filters now support empty / not empty options
  • The Quick Scan dialog has had a major overhaul to make it usable, if not useful. While currently a work in progress, it now offers the following features [#261]:
  • The ability to set a limit on pages per domain during the quick scan
  • A diagram of the scan results is displayed in the dialog using colour coding to show which URI's will be included in a copy, and which will not
  • You can change how you expect the website to be crawled and it will automatically update the diagram to reflect the new setting
  • Using the diagram you can exclude URI's from being crawled, or add excluded domains to be crawled
  • Confirming the dialog no longer resets many settings in your project back to defaults

Changed:

  • The Rules, Forms and Password list editors now share a common base and are now consistent in how to add and edit items
  • You can now re-order rules, forms and passwords in their respective editors by dragging items in the list
  • Rules no longer stop executing after the first match is found, but continue through all rules, allowing for more complex scenarios
  • Rule lists are no longer sorted by default making it easy to see the execution order
  • When calling the CLI to download a single file and the /o argument points to an existing directory, the CLI will generate a filename based on the URI to download [#250]
  • When trying to copy a website, custom expressions are now validated and the copy will not commence if any are invalid
  • The Crawl Content rule flag can now be set independently of the Exclude flag. This finally allows you to create a copy job that will scan an entire website, but only keep files such as images
  • Documentation updates

Deprecated:

  • The Reverse and Do not allow children to inherit this rule rule flags are deprecated and will be removed in a future version of WebCopy

Fixed:

  • Redirects with a relative Location header could be incorrectly combined into absolute URI's
  • Empty analytics sessions are no longer transmitted
  • Failure to obtain shell icons should no longer crash the application
  • Loading a diagram didn't update UI state correctly
  • Changing some diagram properties didn't cause the diagram to be updated
  • URI's which had a blank charset attribute in the Content-Type header weren't processed properly
  • Fixed a crash which could occur using the CLI trying to open a file that wasn't a WebCopy project [#253]
  • Reordering rules and forms didn't reflect properly in the user interface
  • Application no longer crashes if there is an issue exporting or copying large diagram images [#262]
  • CLI will no longer attempt to download if the output folder is protected [#249]
  • When using custom xpath expressions, multiple expressions would be incorrectly created if the same attribute was listed multiple times
  • Several more list views have been virtualized [#64, #65]
  • The rule editor no longer tries to convert patterns into URI's
  • Cloning a WebCopy project skipped numerous values
  • The keep alive setting wasn't persisted correctly
  • Fixed an issue where the Quick Scan dialog could crash with a duplicate key error [#251]
  • Failure to generate the website diagram is no longer fatal [#247]
  • Website diagrams are now generated directly from link information, rather than building a sitemap and generating from that - this should reduce memory requirements of creating the diagram [#247]
  • Form and rule lists should now correctly update if their respective contents change
  • The main results list view is now virtual which should resolve all memory issues relating to working with URI lists [#64, #65]
  • The Capture Form tool will no longer crash if there is a problem creating the embedded browser
  • Pressing Escape in the Capture Form tool no longer closes the window