copy reparse points as is -- not their contents

Here you can propose new features, make suggestions etc.

Moderators: white, Hacker, petermad, Stefan2

mastabog
Junior Member
Junior Member
Posts: 72
Joined: 2004-03-07, 21:36 UTC

copy reparse points as is -- not their contents

Post by *mastabog »

This is something I wanted to suggest for a while. NTFS now support symlinks, hard links and junction points. It would be very useful to have an option in the Copy window of TC that allows the copying/moving of these reparse points as is, and not their contents.

This should work for both when copying/moving the reparse point itself, and when copying a directory that contains reparse points.

Currently TC parses them and copies the contents, which is not always desirable. In my case, it usually isn't ... I even have back-loop junction points in addition to reparse points to some huge folders.

Cheers
User avatar
petermad
Power Member
Power Member
Posts: 14826
Joined: 2003-02-05, 20:24 UTC
Location: Denmark
Contact:

Post by *petermad »

Support++
License #524 (1994)
Danish Total Commander Translator
TC 11.03 32+64bit on Win XP 32bit & Win 7, 8.1 & 10 (22H2) 64bit, 'Everything' 1.5.0.1371a
TC 3.50 on Android 6 & 13
Try: TC Extended Menus | TC Languagebar | TC Dark Help | PHSM-Calendar
User avatar
LonerD
Senior Member
Senior Member
Posts: 381
Joined: 2010-06-19, 20:18 UTC
Location: Makeyevka, Russia
Contact:

Post by *LonerD »

Support for TC native realisation of this option.
And copying-movng between different disks with saving symlink structure - too.

Smart Copy in LSE can do it, but it integrate to Explorer and very uncomfortable.
mastabog
Junior Member
Junior Member
Posts: 72
Joined: 2004-03-07, 21:36 UTC

Post by *mastabog »

There are quite a few apps that have this option, especially among the sync type ones.

Fast Copy is one that I use particularly for this purpose as it supports an extensive command line set of options and can be scripted or even implemented in TC as a menu button to act on the highlighted files and target directory.

There is also xxcopy.
Sob
Power Member
Power Member
Posts: 941
Joined: 2005-01-19, 17:33 UTC

Post by *Sob »

I don't need it very often, but everytime I do, my love for TC drops down considerably. Usually it's just few links, so I didn't bother to find any specialized copying tool yet (and I don't even want to, I like TC as my file manager) and just create the links in new location manually. And TC not being able to help me with that either, does not make the whole experience any better. Modern file manager should really support these things.
User avatar
MaxX
Power Member
Power Member
Posts: 1029
Joined: 2012-03-23, 18:15 UTC
Location: UA

Post by *MaxX »

Support!!! ++++++
Ukrainian Total Commander Translator. Feedback and discuss.
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48118
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

The problem is that you can't know whether the targets of these links will be copied too!
Author of Total Commander
https://www.ghisler.com
Sob
Power Member
Power Member
Posts: 941
Joined: 2005-01-19, 17:33 UTC

Post by *Sob »

I don't think you really have to know that. It's user's responsibility to know what he wants to do with them, even if it might produce wrong results.

I'm far from being an authority on links, but it seems to me that you just need three options what to do with them:

a) copy link target and create it in destination as regular file or directory (as TC does now)
b) copy link as is (relative links might break, but it's user's choice)
c) copy link and adjust target of destination link to point to the same location as original (for relative links)

These three choices are valid for relative symbolic links. For absolute symbolic links, mount points and directory junctions, b) and c) should be the same thing, because those are not relative. With all of these there should be no problem. With hard links it's a little worse, because they can't cross partitions.
User avatar
Balderstrom
Power Member
Power Member
Posts: 2148
Joined: 2005-10-11, 10:10 UTC

Post by *Balderstrom »

TC's handling of Junctions has been questionable since Junctions were introduced (Win2K, I believe).
2005: Flint: [Req] Improvement in work with directory junctions on NTFS.
2007: Status Quo: Junctions not recognized on Move, source contents deleted
2012: zupermario: Do not follow NTFS links
-- Similiar to my request in the thread from 2007.

This request, similiar to my request in zupermario's thread from this year.

I cringe everytime I have to copy folders with TC. Folder structures will get reamed, unneccessary files will be moved/copied, and junctions will be converted to folders.

At least when MOVING folders, junctions inside folders appear to be preserved (Similiar to Explorer).
When copying Folders, junctions are converted to Folders, and their contents duplicated. Unfortunately, even explorer mishandles this IMHO :: does not copy the contents of the junction, but converts it to a normal folder.

If I don't remember that a folder structure contained junctions, and don't manually fix it, and happen to delete the Original Source later... I no longer have junction references pointing to where they are supposed to be, content has been duplicated, and files are no longer referencing the drive/data/content that they are supposed to be.
Sob
Power Member
Power Member
Posts: 941
Joined: 2005-01-19, 17:33 UTC

Post by *Sob »

It's also true that not all aspects of various kinds of links are completely trouble-free. If it was only what would make me happy (those three options in my previous post, I'm modest), then it would be simple. But the fact is, it does not really cover everything.

E.g. copying C:\Users to D: and changing all contained junctions to point also to dirs on D: would require special handling, otherwise they would still point to dirs on C:. I can imagine it in case the whole directory was copied at once, it could use simple "if it points to something inside, then change it, otherwise leave it" rule. But that could also produce unwanted results for those links pointing outside. So for this there could be another option to remap drive letter in targets of absolute links. But the whole thing is getting a little complicated.

Anyway, it would be nice to start with at least something. If I can move the link/junction and it works fine, I'd like to be able to copy it too.
User avatar
Balderstrom
Power Member
Power Member
Posts: 2148
Joined: 2005-10-11, 10:10 UTC

Post by *Balderstrom »

It is somewhat complicated - depending on what you need to do...

Except, Hermann Schinagl has pretty much every usage case you can think of (hardlinks, junctions, symlinks) documented and detailed with examples and screenshots - along with a tool to facilitate all those cases.

Link Shell Extension

Replacement (Junction,Symbolic Link,Mountpoint)
Link Shell Extension can change the target of an existing Junction, Symbolic Link or MountPoint either via Pick/Drop or Drag and Drop.
Smart Copy
Smart Copy basically creates a copy of the directory structure from the source location to the destination, but it preserves the inner hardlink structure and inner junction/symbolic link relations of the source, and recreates this inner hardlink structure and inner junction/symbolic link relation at the destination location.
Crop/Unroll/Splice, Smart Move, Clone, Smart Mirror, Delorean Copy...


Whether Mr.Ghisler would want to do any of that from scratch, or whether Hermann would offer some kind of license for distribution with TC - I couldn't say. I can say TC would benefit significantly from having that functionality built into it's copy/move routines though. Instead of the current situation that passingly automatically handles junctions without any options at all on how they should be handled.
mastabog
Junior Member
Junior Member
Posts: 72
Joined: 2004-03-07, 21:36 UTC

Post by *mastabog »

ghisler(Author) wrote:The problem is that you can't know whether the targets of these links will be copied too!
So is that a no? I would be surprised.

I don't think that's a problem. I have been using Fast Copy (see link above) with great success for a long time. It doesn't seem like it has that much of a problem dealing with links ...

I vote for recreating the link. The target stays exactly the same (no interpretation or mangling). That means that links to absolute paths will still work and links to relative paths will work in many cases. If a link no longer works afterwards because the copy/move operation altered the link's target, then tough luck. That's the user's problem! This is the case in Linux too, and copying links has been a feature for the longest time.

If the user forgot he had links to relative paths when moving/copying and they no longer work afterwards then it's their problem, not Linux's and neither the copying/moving algorithm. Working with links in Linux has been essential since pretty much its inception.

This feature is becoming essential in Windows too. I'm growing dependent on links more and more often in Windows. Even MS is now using all sort of links by default in their OSs (see Win7 for instance).
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48118
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

Imagine the following situation: The user selects two folders,
folderwithlinks
and
folderwithlinktargets

and tries to copy them. The first contains links to files within folderwithlinktargets. To copy the links correctly, TC would have to first copy the files within folderwithlinktargets, and later the links themselves within folderwithlinks. Now there are multiple problems with this scenario:
1. The user may abort the copying in the middle. Currently you can see from the progress dialog how far the program could copy. With the new method, all the links would be missing from the otherwise already copied folders. The user couldn't easily resume the copying.
2. folderwithlinktargets may already exist in the target, and the user may choose to skip some files when overwriting. Then the copied links may point to this older data, while the user expects to see the newer data from the source.
3. The user may have set a filter, e.g. *.txt, and the links may be pointing to files which are not copied, while the links themselves (which may have a different extension) may have to be copied.
There are certainly many more similar problems.
Author of Total Commander
https://www.ghisler.com
User avatar
Balderstrom
Power Member
Power Member
Posts: 2148
Joined: 2005-10-11, 10:10 UTC

Post by *Balderstrom »

There aren't many more similiar problems.
There are basically two types of junctions/symlinks:
1) Inner Links
2) Outer Links

An Inner Link references within the "root" of the top-folder being copied/moved.
Whereas an Outer Link references outside of the "root".

These inner/outer links can themselves also be inner/outer with respect to where they reference.

When dealing with Links (Junctions/Symlinks),
1) Build the directory structure with normal folders.
2) Note Symlinked (or Hardlinked) files.
3) Check for any conflicts with said files.
-- Be that duplicate names in target or
-- Hardlinks that would traverse outside the Drive in question.

Allow the user to choose,
1) An overriding default behaviour for dealing with these conflicts.
2) Ask the user what to do with the conflicts, one by one.

While processing said conflicts:
1) Log the conflicts that arise, and if they are automatically handled:
2) Notify the user what was done:
---> Link was broken, and not copied.
---> Link was traversed, and duplicate non-link was created.

It doesn't matter what the user expects --- so long as the user is notified about HOW the conflicts were handled.

Note, TC shouldn't just notify the user of a NAME conflict, when one or more of the files are Links, it should also notify that the source in question (or target) is a link.
TC could also notify which is the newer file - be that a link source, link target. As well as indicating whether a given action will duplicate content, or break a link reference.

Currently we just wind up with duplicated content and no option whatsoever. You are trying to claim that due to the possibility that something may not occur like the user expects then nothing should be done at all. That is a fallacious argument.
2. folderwithlinktargets may already exist in the target, and the user may choose to skip some files when overwriting. Then the copied links may point to this older data, while the user expects to see the newer data from the source.
This makes no sense to me. If a file is symlinked it points to specific file/data; similiarly for a hardlink (that can't span drives).
If a symlink exists in the target and is NOT overwritten by a file with the same name, it will continue to reference what it did before.
How is that unexpected behaviour? And how is that not what the user would expect? You skipped copying the file, it remains what it was before. How can the user expect to see newer data from "source" if they skipped copying it?
User avatar
ghisler(Author)
Site Admin
Site Admin
Posts: 48118
Joined: 2003-02-04, 09:46 UTC
Location: Switzerland
Contact:

Post by *ghisler(Author) »

Regarding your last comment: Imagine this situation:

folderwithlinks\partlist.xls
pointing to
folderwithlinktargets \partlist.xls

The user copies folderwithlinks\partlist.xls to the backup disk, and expects that the newly added parts in the list appear in the copied file. However, folderwithlinks\partlist.xls is just a link to folderwithlinktargets \partlist.xls. Since the latter isn't copied, the backup disk still only contains the older folderwithlinktargets \partlist.xls, so the copied link still points to the older file.

But the user sees the new data when double clicking on folderwithlinks\partlist.xls on the source side, and expects to see the new data also after copying that "file"...
Author of Total Commander
https://www.ghisler.com
Post Reply