[john-users] Advise on best approach (truecrypt pw based on pdf file)

[Demian Smith]

Hi,

I've recently lost my truecrypt PW and am thinking to approach the
recovery with JtR but am unsure about the best approach to do so. So
hopefully I could get a couple of pointers into the correct direction of
setting it up.

I know a couple of facts, which hopefully make it feasible in the first
place:
- I have created the password from a random page of a random book¹
- First letter MUST be capital
- No numbers or diacritics,
- Likely only I as second capital letter
- No digits
- Probably 5 -12 chars (I imagine it to be around 8, but that's guessing)

At the moment I have tried creating a fake pot file from ¹ and creating
a chr of it and then building a incremental rule:

[Incremental:rc]
File = $JOHN/rc.chr
MinLen = 5
MaxLen = 12
CharCount = 36

And have now tried up till now
0g 7093116p 2:14:48:30 0g/s 31.37p/s 62.74c/s 62.74C/s toisooaaa..toisooatc


However, I feel like wasting loads of work and time with tries I don't
need (whitespaces - which I thought I had removed -, non-capital letters
as first letter...) and I assume there must be a more efficient way of
attempting my recovery.

I am running
1.8.0.2-bleeding-jumbo_mpi+omp [linux-gnu 64-bit SSE4.1-autoconf]
on a Intel(R) Core(TM) i5 CPU M 480 @ 2.67GHz

So, can I please ask your advise on how to apply a decent statistical
analysis on the txt of the book and/or prepare a proper
incremental/external rule or, simply speaking, what's the best approach
in your opinions?

(¹ I have tried
cat INFILE | sed -re 's/[?!.:]/\n/g' | sed -re
's/([A-Za-z])[A-Za-z]+[^A-Za-z]*/\1/g' > outfile
on a txt version of the pdf and using "outfile" as a wordlist - no luck)

Kind regards and thank you in advance,
Demian

[magnum]

Post by Demian Smith
I've recently lost my truecrypt PW and am thinking to approach the
recovery with JtR but am unsure about the best approach to do so. So
hopefully I could get a couple of pointers into the correct direction of
setting it up.
I know a couple of facts, which hopefully make it feasible in the first
- I have created the password from a random page of a random book¹
- First letter MUST be capital
- No numbers or diacritics,
- Likely only I as second capital letter
- No digits
- Probably 5 -12 chars (I imagine it to be around 8, but that's guessing)

Are you saying the password is made up from initial letters of a
sentence? As in "To be or not to be, that's the question" -> "Tbontbttq".

Post by Demian Smith
At the moment I have tried creating a fake pot file from ¹ and creating

(...)

Post by Demian Smith
(¹ I have tried
cat INFILE | sed -re 's/[?!.:]/\n/g' | sed -re
's/([A-Za-z])[A-Za-z]+[^A-Za-z]*/\1/g' > outfile

Did you look at outfile and see if your sed magic does everything you
thought? You should probably just adjust it a little.

magnum

[magnum]

Post by magnum

Post by Demian Smith
I've recently lost my truecrypt PW and am thinking to approach the
recovery with JtR but am unsure about the best approach to do so. So
hopefully I could get a couple of pointers into the correct direction of
setting it up.
I know a couple of facts, which hopefully make it feasible in the first
- I have created the password from a random page of a random book¹
- First letter MUST be capital
- No numbers or diacritics,
- Likely only I as second capital letter
- No digits
- Probably 5 -12 chars (I imagine it to be around 8, but that's guessing)

Are you saying the password is made up from initial letters of a
sentence? As in "To be or not to be, that's the question" -> "Tbontbttq".

Trying your sed magic, this seems to be the case. But did you use the
first sentence on a page, or a random one?

Post by magnum

Post by Demian Smith
At the moment I have tried creating a fake pot file from ¹ and creating

(...)

Post by Demian Smith
(¹ I have tried
cat INFILE | sed -re 's/[?!.:]/\n/g' | sed -re
's/([A-Za-z])[A-Za-z]+[^A-Za-z]*/\1/g' > outfile

Did you look at outfile and see if your sed magic does everything you
thought? You should probably just adjust it a little.

Your version did not remove all spaces and did also not handle
one-letter words like "I" or "a". This seems to work better:

sed -re 's/[?!.:,;"]+ ?/\n/g' | sed -re
's/([A-Za-z])[A-Za-z]*[^A-Za-z]*/\1/g' | grep -E '^[A-Z]'

It's still flawed because it keeps the original linefeeds, breaking
sentences. I'd do it in Perl instead. Actually if you google a little
I'd be surprised if you can't find a perfected perl script ready to use!

magnum

[Frank Dittrich]

Post by Demian Smith
I know a couple of facts, which hopefully make it feasible in the first
- I have created the password from a random page of a random book¹
- First letter MUST be capital
- No numbers or diacritics,
- Likely only I as second capital letter
- No digits
- Probably 5 -12 chars (I imagine it to be around 8, but that's guessing)

[...]

Post by Demian Smith
(¹ I have tried
cat INFILE | sed -re 's/[?!.:]/\n/g' | sed -re
's/([A-Za-z])[A-Za-z]+[^A-Za-z]*/\1/g' > outfile
on a txt version of the pdf and using "outfile" as a wordlist - no luck)

Could your txt verskion have line breaks in other places than the pdf
version?

Frank

[Demian Smith]

Hi Magnum, Hi Frank,

Are you saying the password is made up from initial letters of a sentence? As in "To be or not to be, that's the question" -> "Tbontbttq".

Exactly this, yes. Not the smartest approach in terms of "memorable",
but I did not think that I'd lose my HDD in an unrecoverable state.

Trying your sed magic, this seems to be the case. But did you use the first sentence on a page, or a random one?

It had bee na random one - initially I thought I'd new the page (as it
had potentially been opened on a different machine which saved the page
last opened. None of the sentences there worked, which, in my layman’s
thoughts indicates it had been a different page or I made a typo...

sed -re 's/[?!.:,;"]+ ?/\n/g' | sed -re 's/([A-Za-z])[A-Za-z]*[^A-Za-z]*/\1/g' | grep -E '^[A-Z]'

This looks really nice - while I'm using cl every now and then, sed
magic like this is way over my head, so thank you very much!

It's still flawed because it keeps the original linefeeds, breaking sentences.
Could your txt verskion have line breaks in other places than the pdf
version?

Looking over a couple of pages it seems as if pdftotext is doing a
decent enough job. But I have gone a different route there and have just
removed all of the linebreaks with
{ tr '\n' ' ' < mytext.txt; echo; } | sed '$s/ $//'
before doing the other sed magic. Partly because I might as well have
used only half a sentence (up to a , or an "and") . it had bee na
sentence which made grammatical sense...

So, after a bit of fine tuning - removing all linebreaks, keeping only
first letters of any words, removing whitespaces (simply in kate, to be
honest), converting . to linebreaks and removing all lines starting with
lowercase letters I had a hopefully decent txt file. This I fed into my
"adoption" of the Word List Cleaner and made a fake pot file out of it,
which I then used to create a chr.

0g 23000p 0:00:09:18 0g/s 41.15p/s 82.31c/s 82.31C/s Tfneatwa..Tfneotps
0g 25868p 0:00:13:15 0g/s 32.50p/s 65.00c/s 65.00C/s Iceuaeog..Iceuabab

So it looks -to me - as if I am on the right track, mainly due to
--make-charset and incremental mode are doing a really great job there.

Thank you ever so much, lads, I really appreciate your help and thoughts
on this!
Now I'm just hoping that luck will be on my side and I'll recovery it
within reasonable time and before my cores are burning out ...

Thanks again,
Demian

★ On 15/05/08 10:39 p.m. Frank Dittrich wrote ★

Post by Demian Smith
I know a couple of facts, which hopefully make it feasible in the first
- I have created the password from a random page of a random book¹
- First letter MUST be capital
- No numbers or diacritics,
- Likely only I as second capital letter
- No digits
- Probably 5 -12 chars (I imagine it to be around 8, but that's guessing)

[...]

Post by Demian Smith
(¹ I have tried
cat INFILE | sed -re 's/[?!.:]/\n/g' | sed -re
's/([A-Za-z])[A-Za-z]+[^A-Za-z]*/\1/g' > outfile
on a txt version of the pdf and using "outfile" as a wordlist - no luck)

Could your txt verskion have line breaks in other places than the pdf
version?
Frank

[Frank Dittrich]

Post by Demian Smith
So, after a bit of fine tuning - removing all linebreaks, keeping only
first letters of any words, removing whitespaces (simply in kate, to be
honest), converting . to linebreaks and removing all lines starting with
lowercase letters I had a hopefully decent txt file. This I fed into my
"adoption" of the Word List Cleaner and made a fake pot file out of it,
which I then used to create a chr.

0g 23000p 0:00:09:18 0g/s 41.15p/s 82.31c/s 82.31C/s Tfneatwa..Tfneotps
0g 25868p 0:00:13:15 0g/s 32.50p/s 65.00c/s 65.00C/s Iceuaeog..Iceuabab

Did you try that new word list first? You should.

Frank

[magnum]

Post by Demian Smith

0g 23000p 0:00:09:18 0g/s 41.15p/s 82.31c/s 82.31C/s Tfneatwa..Tfneotps
0g 25868p 0:00:13:15 0g/s 32.50p/s 65.00c/s 65.00C/s Iceuaeog..Iceuabab

So it looks -to me - as if I am on the right track, mainly due to
--make-charset and incremental mode are doing a really great job there.

Good, candidates look OK now. We still have two issues:

Even if incremental now runs what you intended, it could take a LONG
time (as in years) to hit the correct phrase. What if you use that
wordlist you produced and instead of making a fake pot file, use it
directly as a wordlist in a crack? If it's the correct book I reckon
you'd crack your password in no time, no?

Also, speed doesn't look too good. It seems to run slow. Or maybe our
test vectors doesn't reflect real-world costs.

Are you running without OpenMP, or on a CPU with just one core? Please
stop this job (as you probably know it can be resumed) and post the
benchmark figures from:

$ ../run/john -test -form:tc_*

Also, when you resume the job, copy the inital lines output when job
starts and report them too; Is it loading two "hashes"? Shouldn't it be
just one?

magnum

[magnum]

Post by Demian Smith
I am running
1.8.0.2-bleeding-jumbo_mpi+omp [linux-gnu 64-bit SSE4.1-autoconf]

It looks to me you should upgrade to latest version from GitHub. Jim did
some significant speedup in Truecrypt (if you are lucky enough the right
algo is used).

Also, do not use MPI unless you really need it (that is, run a cluster
over the network). Most people do better using --fork.

magnum

[Demian Smith]

Hi again,

Post by Frank Dittrich
Did you try that new word list first? You should.
Even if incremental now runs what you intended, it could take a LONG time (as in years) to hit the correct phrase. What if you use that wordlist you produced and instead of making a fake pot file, use it directly as a wordlist in a crack? If it's the correct book I reckon you'd crack your password in no time, no?

I should have, indeed - I just did, but no luck. Potentially due to a
typo or, more likely, a phrase compiled out of only a partial sentence
in the book...

I was hoping in the beginning it would crack with a word list, but it
looks like I am out of luck in this regard...

Post by Frank Dittrich

Post by magnum
It looks to me you should upgrade to latest version from GitHub. Jim did
some significant speedup in Truecrypt (if you are lucky enough the right
algo is used)

Oh, last time I checked (a while ago) this had been the newest. I'm
downloading the newer one right now.

Post by Frank Dittrich

Post by magnum
Also, do not use MPI unless you really need it (that is, run a cluster
over the network). Most people do better using --fork.

I think MPI is there from my first autoconfig and only for hashes that
don't support OMP (in case I ever would need it). I run john with an alias:
alias john='OMP_NUM_THREADS=4 ~/.bin/JtR/run/john'

My benachmark looks like this (with the latest bleeding from github
which I downloaded and made in the meantime)
initUnicode(UNICODE, ASCII/ASCII)
ASCII -> ASCII -> ASCII
Will run 4 OpenMP threads
Benchmarking: tc_aes_xts, TrueCrypt AES256_XTS [SHA512 128/128 SSE4.1 2x
/RIPEMD160/WHIRLPOOL]... (4xOMP) DONE
Speed for cost 1 (hash algorithm [1:SHA512 2:RIPEMD160 3:Whirlpool]) of 1
Raw: 2112 c/s real, 550 c/s virtual

Benchmarking: tc_ripemd160, TrueCrypt AES256_XTS [RIPEMD160 32/64]...
(4xOMP) DONE
Raw: 472 c/s real, 117 c/s virtual

Benchmarking: tc_sha512, TrueCrypt AES256_XTS [SHA512 128/128 SSE4.1
2x]... (4xOMP) DONE
Raw: 2096 c/s real, 547 c/s virtual

Benchmarking: tc_whirlpool, TrueCrypt AES256_XTS [WHIRLPOOL 64/64]...
(4xOMP) DONE
Raw: 696 c/s real, 174 c/s virtualAll 3 formats passed self-tests!

(I ran the benachmark with the older version of John and boy is there a
difference)

Post by Frank Dittrich
Also, when you resume the job, copy the inital lines output when job starts and report them too; Is it loading two "hashes"? Shouldn't it be just one?

It does load 2 hashes alright, not sure what the reason would be, though?

version 1.8.0.4-jumbo-1-bleeding_omp [linux-gnu 64-bit SSE4.1-autoconf]

Post by Frank Dittrich
Loaded 2 password hashes with 2 different salts (tc_ripemd160,

TrueCrypt AES256_XTS >[RIPEMD160 32/64])

Post by Frank Dittrich
Will run 4 OpenMP threads
Press 'q' or Ctrl-C to abort, almost any other key for status
0g 3178604p 0:19:58:10 0g/s 44.21p/s 88.42c/s 88.42C/s AioseaaU

Would it be better to double check the hash file and maybe create it
anew? Or is it a mal configuration of john.conf ?

Again, thank you so much guys for your help and support!

Demian

[magnum]

Post by Demian Smith

Post by magnum
Even if incremental now runs what you intended, it could take a
LONG time (as in years) to hit the correct phrase. What if you use
that wordlist you produced and instead of making a fake pot file,
use it directly as a wordlist in a crack? If it's the correct book
I reckon you'd crack your password in no time, no?

I should have, indeed - I just did, but no luck. Potentially due to a
typo or, more likely, a phrase compiled out of only a partial sentence
in the book...

If that's it, we can create permutation rules tailored for this and
should be able to crack it. I think you should concentrate on this for a
while before resuming that incremental run.

Post by Demian Smith
I ran the benachmark with the older version of John and boy is there a
difference

Great! This is still a very tough hash type though.

Post by Demian Smith

Post by magnum
Also, when you resume the job, copy the inital lines output when
job starts and report them too; Is it loading two "hashes"?
Shouldn't it be just one?

It does load 2 hashes alright, not sure what the reason would be, though?

I'm not familiar with TC so I can't tell if you should filter one out or
not. Let's keep attacking both for now.

Post by Demian Smith
Loaded 2 password hashes with 2 different salts (tc_ripemd160, TrueCrypt AES256_XTS >[RIPEMD160 32/64])
Will run 4 OpenMP threads
Press 'q' or Ctrl-C to abort, almost any other key for status
0g 3178604p 0:19:58:10 0g/s 44.21p/s 88.42c/s 88.42C/s AioseaaU

Note that it still says 44.21p/s here but that's just the average speed
from job start so this figure will now increase slowly towards your new
faster figure.

Post by Demian Smith
Would it be better to double check the hash file and maybe create it
anew? Or is it a mal configuration of john.conf ?

Let's ignore that for now (unless someone can chime in with knowledge).

Let's concentrate on wordlist rules. If it's indeed a partial sentence,
would you think it's truncated in the end and still starting with an
uppercase letter?

Also, how large (number of lines) is that wordlist? Did you try just
running it with --rules (and perhaps --rules=single in another run)?

magnum

[Frank Dittrich]

Post by Demian Smith
I've recently lost my truecrypt PW and am thinking to approach the
recovery with JtR but am unsure about the best approach to do so. So
hopefully I could get a couple of pointers into the correct direction of
setting it up.

Is that a truecrypt encrypted Windows System partition, or just a normal
partition or truecrypt volume?
I just ask because I am not sure whether anybody verified that using
john for Windows system partitions works at all.
(I think I remember that the realcrypt Linux package had trouble with
Windows system partitions, but I don't remember any details.)

That truecrypt2john was able to extract something that it interpreted as
a truecrypt partition "header" or whatever does not really mean that
you'll notice when you enter the correct password.

So, we need someone who is willing to share some information about a
real Windows system partition including the clear text password.
Then we need to check whether john finds the correct known password for
the hash(es?) extracted by truecrypt2john or whether some adjustments
are needed.
If you create a new Windows test installation for that purpose, make
sure you don't use a GPT, because truecrypt can't handle GPTs.
May be https://github.com/DrWhax/truecrypt-archive could help to find
out whether and how encrypted Windows system partitions differ from
regular partitions or truecrypt volumes.

Frank

[Frank Dittrich]

Post by Frank Dittrich
So, we need someone who is willing to share some information about a
real Windows system partition including the clear text password.
Then we need to check whether john finds the correct known password for
the hash(es?) extracted by truecrypt2john or whether some adjustments
are needed.

If someone with a truecrypt encrypted Windows system partition can
verify that the known password works when running truecrypt2john and
john, then there is obviously no need to share any more details than
just confirming that it worked.

Frank

[Demian Smith]

Hi,

Post by Frank Dittrich
If someone with a truecrypt encrypted Windows system partition can
verify that the known password works when running truecrypt2john and
john, then there is obviously no need to share any more details than
just confirming that it worked.
Is that a truecrypt encrypted Windows System partition, or just a normal
partition or truecrypt volume?

In this case it's not a windows partition but an external RAID1, fully
disk encryption with truecrypt. I did create a full encrypted second
device (USB flash key) and truecrypt2john was extracting hashes and did
find the correct "guess2 from a wordlist. However, the RAID had been
encrypted on Windows, whereas the flash key has been done on linux.
Should this matter?

Post by Frank Dittrich
If that's it, we can create permutation rules tailored for this and should be able to crack it. I think you should concentrate on this for a while before resuming that incremental run.

Thanks for the pointer, I will look into these - I did not yet use them
at all up till now.

Post by Frank Dittrich
Note that it still says 44.21p/s here but that's just the average speed from job start so this figure will now increase slowly towards your new faster figure.

I was wondering about this, but (naturally) you're right and the number
keeps slowly climbing up.

Post by Frank Dittrich
Let's concentrate on wordlist rules. If it's indeed a partial sentence, would you think it's truncated in the end and still starting with an uppercase letter?

It does start with a upper case letter, that's for sure. If I understand
correctly, it would have been truncated at the end - if the original
sentence would have been

Post by Frank Dittrich
To be or not to be, that is the question, and indeed a question which

shall

Post by Frank Dittrich
be answered, by this circle of fine man who gathered on this moonlit

December night

It would be any of these
Tbontb
Tbontbtistq
Tbontbtistqaiaqwsba
Tbontbtistqaiaqwsbabtcofm
and so on.

I have tried replacing "and" with a fullstop to force a linebreak with
sed, but this would not work if it would have been two ands in a
sentence or a comma ...

Post by Frank Dittrich
Also, how large (number of lines) is that wordlist? Did you try just running it with --rules (and perhaps --rules=single in another run)?

The list I have created by removing all lines starting with non capital
letters is a whopping 3803 lines long, but the last lines are in the
ranks of 50 and more chars (the longest being 97 chars).

After your pointer towards rules I have now cut the lines at 20 chars
and am running it with the rules on the wordlist. I've not worked with
rules either, would I have to write some useful ones (for my case) first?

Cheers,
Demian

[magnum]

Post by Demian Smith

Post by magnum
Let's concentrate on wordlist rules. If it's indeed a partial
sentence, would you think it's truncated in the end and still
starting with an uppercase letter?

It does start with a upper case letter, that's for sure. If I understand
correctly, it would have been truncated at the end - if the original
sentence would have been

Post by magnum
To be or not to be, that is the question, and indeed a question which shall
be answered, by this circle of fine man who gathered on this moonlit December night

It would be any of these
Tbontb
Tbontbtistq
Tbontbtistqaiaqwsba
Tbontbtistqaiaqwsbabtcofm
and so on.

So you wouldn't stop in the middle of a sub phrase (or whatever it's
called) ie. you wouldn't have used "Tbontbti"? That's hard to automate
so we'll probably just have to ignore it and try any truncations to eg.
5-12 characters.

Post by Demian Smith

Post by magnum
Also, how large (number of lines) is that wordlist? Did you try
just running it with --rules (and perhaps --rules=single in another
run)?

The list I have created by removing all lines starting with non capital
letters is a whopping 3803 lines long, but the last lines are in the
ranks of 50 and more chars (the longest being 97 chars).

OK that's a small amount. We should be able to run that through a decent
number of permutations if needed.

Post by Demian Smith
After your pointer towards rules I have now cut the lines at 20 chars
and am running it with the rules on the wordlist. I've not worked with
rules either, would I have to write some useful ones (for my case) first?

Here's a quick'n'dirty one:

[List.Rules:truncate]

Post by Demian Smith
[5-9A-C]'\0

Put the above in john.conf and use it with "--rules=truncate". For an
input word of Tbontbtistqaiaqwsbabtcofm it will output these 8 variations:

Tbont
Tbontb
Tbontbt
Tbontbti
Tbontbtis
Tbontbtist
Tbontbtistq
Tbontbtistqa

If you want longer than 12, change the 'C' in A-C. 5-9 means literally
5-9, A means 10, B is 11 and so on so A-C here means a truncate at a
length of 10-12. I think you can actually use A-Z in this case and it
wont take too long.

magnum

[Demian Smith]

Hi Magnum,

thanks for the truncate rule, I am running these now on the File
generated, I might even (if it's not successful) on a bigger-ish file
with the non-capital sentences as well.

I have taken your advise and am running it with A-Z and have as well
intentions to look into the "rules" doc to see what 5-9 in this instance
means.

john --wordlist=/home/Demian/RAID/txt/wpd8.txt --rules=truncate
--session=truncate2 /home/Demian/RAID/recover

-> 0g 52648p 0:00:07:24 47.43% (ETA: 18:10:31) 0g/s 118.3p/s 709.9c/s
709.9C/s AEaiitwAhbiapotwc..Awwmgdtcaahriafts

So it might not take to long anyway...

Thanks for your support and for keeping my hopes up =)
Demian
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'

Sinéad O'Connor

★ On 15/05/09 11:37 p.m. Magnum wrote ★

Post by magnum

Post by Demian Smith

Post by magnum
Let's concentrate on wordlist rules. If it's indeed a partial
sentence, would you think it's truncated in the end and still
starting with an uppercase letter?

It does start with a upper case letter, that's for sure. If I understand
correctly, it would have been truncated at the end - if the original
sentence would have been

Post by magnum
To be or not to be, that is the question, and indeed a question which shall
be answered, by this circle of fine man who gathered on this moonlit December night

It would be any of these
Tbontb
Tbontbtistq
Tbontbtistqaiaqwsba
Tbontbtistqaiaqwsbabtcofm
and so on.

So you wouldn't stop in the middle of a sub phrase (or whatever it's
called) ie. you wouldn't have used "Tbontbti"? That's hard to automate
so we'll probably just have to ignore it and try any truncations to eg.
5-12 characters.

Post by Demian Smith

Post by magnum
Also, how large (number of lines) is that wordlist? Did you try
just running it with --rules (and perhaps --rules=single in another
run)?

The list I have created by removing all lines starting with non capital
letters is a whopping 3803 lines long, but the last lines are in the
ranks of 50 and more chars (the longest being 97 chars).

OK that's a small amount. We should be able to run that through a decent
number of permutations if needed.

Post by Demian Smith
After your pointer towards rules I have now cut the lines at 20 chars
and am running it with the rules on the wordlist. I've not worked with
rules either, would I have to write some useful ones (for my case)
first?

[List.Rules:truncate]

Post by Demian Smith
[5-9A-C]'\0

Put the above in john.conf and use it with "--rules=truncate". For an
Tbont
Tbontb
Tbontbt
Tbontbti
Tbontbtis
Tbontbtist
Tbontbtistq
Tbontbtistqa
If you want longer than 12, change the 'C' in A-C. 5-9 means literally
5-9, A means 10, B is 11 and so on so A-C here means a truncate at a
length of 10-12. I think you can actually use A-Z in this case and it
wont take too long.
magnum

[magnum]

Post by Demian Smith
★ On 15/05/09 11:37 p.m. Magnum wrote ★

Post by magnum
[List.Rules:truncate]

Post by Demian Smith
[5-9A-C]'\0

Put the above in john.conf and use it with "--rules=truncate". For an
Tbont
Tbontb
Tbontbt
Tbontbti
Tbontbtis
Tbontbtist
Tbontbtistq
Tbontbtistqa

I have taken your advise and am running it with A-Z and have as well
intentions to look into the "rules" doc to see what 5-9 in this
instance means.

There's nothing special about 5-9, 5 means 5 and 9 means 9. The special
is rather that that A means 10 and so on.

Here's a simple rule that truncates to length 5:

'5

Here's one that's enhanced to skip words that weren't at least length 5

Post by Demian Smith
5'5

The >5 means that the rule only applies to words longer than 5
characters, and the '5 is the actual truncation.

The bracket thing is pre-processor stuff. [5-9A-C] will expand to one
rule with that whole bracket string replaced with "5", another line
using "6", and so on. And the \0 means "repeat the last bracket
expansion". So our pre-processor rule of

Post by Demian Smith
[5-9A-C]'\0
5'5
6'6
7'7
8'8
9'9
A'A
B'B
C'C

So if you look in the log file, it should say that your one-line "rule"
is actually 8 rules after pre-processing. Or in case of 5-9A-Z, 31 rules
(covering lengths 5 through 36, save for fence-post errors).

magnum

[Demian Smith]

Hi Magnum,

thanks for your ongoing patience with a young Padawan :s

I am kind of lost, in my opinion the truncate should have easily cracked
the hash, but to no avail. I ran your sed magick again over the txt file
(without linebreaks) and used the truncate rule on it, no luck.

wordfile had 10686 lines and required 85488 bytes for index.
0:00:00:00 - suppressed 1 duplicate lines and/or comments from wordlist.
0:00:00:00 - 31 preprocessed word mangling rules

So I am afraid I did something utterly stupid (like starting with the
last art of a sentence and making it a new sentence, typos, adding a
number at the beginnin or end - which I doubt - et cetera) and hence
will have to go back to incremental and keeping fingers crossed. Unless
someone else has another good idea ...

It is anyway good to start learning about the rules, in case I ever need
them again :)

Cheers,
Demian
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'

Sinéad O'Connor

★ On 15/05/10 06:24 p.m. Magnum wrote ★

Post by Demian Smith
★ On 15/05/09 11:37 p.m. Magnum wrote ★

Post by magnum
[List.Rules:truncate]

Post by Demian Smith
[5-9A-C]'\0

Put the above in john.conf and use it with "--rules=truncate". For an
Tbont
Tbontb
Tbontbt
Tbontbti
Tbontbtis
Tbontbtist
Tbontbtistq
Tbontbtistqa

I have taken your advise and am running it with A-Z and have as well
intentions to look into the "rules" doc to see what 5-9 in this
instance means.

There's nothing special about 5-9, 5 means 5 and 9 means 9. The special
is rather that that A means 10 and so on.
'5
Here's one that's enhanced to skip words that weren't at least length 5

Post by Demian Smith
5'5

The >5 means that the rule only applies to words longer than 5
characters, and the '5 is the actual truncation.
The bracket thing is pre-processor stuff. [5-9A-C] will expand to one
rule with that whole bracket string replaced with "5", another line
using "6", and so on. And the \0 means "repeat the last bracket
expansion". So our pre-processor rule of

Post by Demian Smith
[5-9A-C]'\0
5'5
6'6
7'7
8'8
9'9
A'A
B'B
C'C

So if you look in the log file, it should say that your one-line "rule"
is actually 8 rules after pre-processing. Or in case of 5-9A-Z, 31 rules
(covering lengths 5 through 36, save for fence-post errors).
magnum

[Demian Smith]

Hi list,

still working on my truecrypt hash - in the meantime I have converted
all pdfs in a folder to txt and created the "first letter only" candidates.

I am currently running the truncate rule that magnum thankfully provided
me with on the wordlist (>[4-9A-Z]'\0 )

As this had not been succesful yet even though I assume the correct pdf
had been used I was wondering could a rule be written to do the
"opposite" of truncate?

So instead of truncating
Abcdefgeh to Abcd | Abc | Ab it would start from the back and
"truncates" to fgeh | geh | ge instead. I hope by doing so I fetch out
the stuff I have not tested as yet, in case I started at some random
spot in the sentence...

I am not sure whether this question even makes sense at all, but I am
getting really desperate here, and quickly so...

Thanks for reading,
Marcel


★ On 15/05/10 10:38 p.m. Demian Smith wrote ★

Post by Demian Smith
Hi Magnum,
thanks for your ongoing patience with a young Padawan :s
I am kind of lost, in my opinion the truncate should have easily cracked
the hash, but to no avail. I ran your sed magick again over the txt file
(without linebreaks) and used the truncate rule on it, no luck.

wordfile had 10686 lines and required 85488 bytes for index.
0:00:00:00 - suppressed 1 duplicate lines and/or comments from wordlist.
0:00:00:00 - 31 preprocessed word mangling rules

So I am afraid I did something utterly stupid (like starting with the
last art of a sentence and making it a new sentence, typos, adding a
number at the beginnin or end - which I doubt - et cetera) and hence
will have to go back to incremental and keeping fingers crossed. Unless
someone else has another good idea ...
It is anyway good to start learning about the rules, in case I ever need
them again :)
Cheers,
Demian
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
★ On 15/05/10 06:24 p.m. Magnum wrote ★

Post by Demian Smith
★ On 15/05/09 11:37 p.m. Magnum wrote ★

Post by magnum
[List.Rules:truncate]

Post by Demian Smith
[5-9A-C]'\0

Put the above in john.conf and use it with "--rules=truncate". For an
Tbont
Tbontb
Tbontbt
Tbontbti
Tbontbtis
Tbontbtist
Tbontbtistq
Tbontbtistqa

I have taken your advise and am running it with A-Z and have as well
intentions to look into the "rules" doc to see what 5-9 in this
instance means.

There's nothing special about 5-9, 5 means 5 and 9 means 9. The special
is rather that that A means 10 and so on.
'5
Here's one that's enhanced to skip words that weren't at least length 5

Post by Demian Smith
5'5

The >5 means that the rule only applies to words longer than 5
characters, and the '5 is the actual truncation.
The bracket thing is pre-processor stuff. [5-9A-C] will expand to one
rule with that whole bracket string replaced with "5", another line
using "6", and so on. And the \0 means "repeat the last bracket
expansion". So our pre-processor rule of

Post by Demian Smith
[5-9A-C]'\0
5'5
6'6
7'7
8'8
9'9
A'A
B'B
C'C

So if you look in the log file, it should say that your one-line "rule"
is actually 8 rules after pre-processing. Or in case of 5-9A-Z, 31 rules
(covering lengths 5 through 36, save for fence-post errors).
magnum

[magnum]

Post by Demian Smith
still working on my truecrypt hash - in the meantime I have converted
all pdfs in a folder to txt and created the "first letter only" candidates.
I am currently running the truncate rule that magnum thankfully provided
me with on the wordlist (>[4-9A-Z]'\0 )
As this had not been succesful yet even though I assume the correct pdf
had been used I was wondering could a rule be written to do the
"opposite" of truncate?
So instead of truncating
Abcdefgeh to Abcd | Abc | Ab it would start from the back and
"truncates" to fgeh | geh | ge instead. I hope by doing so I fetch out
the stuff I have not tested as yet, in case I started at some random
spot in the sentence...

Here's a shot from the hip. I half heartedly tossed things together and
it (seemingly) started working much sooner than I expected:

[List.Rules:substring]

Post by Demian Smith
[4-9A-Z] val\0 Xal0 '\0 >[3-9A-Y]'\p[4-9A-Z]

The above expands to 1024 rules. It appears to produce all combinations
of min. length 4 except the full original word. I did not test it other
than the below:

$ ../run/john -pipe -rules:substring -stdout <<< Abcdefgh
efgh
defg
defgh
cdef
cdefg
cdefgh
bcde
bcdef
bcdefg
bcdefgh

magnum

[Demian Smith]

Hi magnum,

thanks a million for the rule Orz !

I ran it over my file, alas - to no avail. I start to think there's
something really wrong/awkward, in all likelihood on my side of thinks
when creating the encrypted HDD.

I have as well created of copy of the wordlist and manually went through
the 3500 lines to check where there wrong line breaks (i.e. lines
starting with lower case).

So, this is my last question and with it I wish to apply for the "Most
stupid question - 2015" award - is it possible, to create a "moving
interval" rule?

I am thinking to take my wordlist rule, remove all the line breaks so
that I have a really long stream of chars and then just move the Interval.

Say, my stream would be
SiadqrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt

I'd like to have rule (or preoprocessed rule) that walks throught the
file like this
[Siad]qrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
[Siadqrfewmnsohabiwto]arotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqrf]ewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
Si[adqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt

This might not result in any positives, but after I have treid all the
other things, I just would like to use this last attempt, before going
back to "BruteForcing" with incremental...

Right now the --rules=single runs on the file, but it takes quite long
as I have the additional ones from KoreLogic in there ( which in my case
probably doesn't make any sense in the first place)

Anyway, if I could get a quick word on my interval idea, I hope I can
then stop asking questions and bug you :s

Thanks,
Demian

★ On 15/05/14 01:32 a.m. Magnum wrote ★

Post by magnum

Post by Demian Smith
still working on my truecrypt hash - in the meantime I have converted
all pdfs in a folder to txt and created the "first letter only" candidates.
I am currently running the truncate rule that magnum thankfully provided
me with on the wordlist (>[4-9A-Z]'\0 )
As this had not been succesful yet even though I assume the correct pdf
had been used I was wondering could a rule be written to do the
"opposite" of truncate?
So instead of truncating
Abcdefgeh to Abcd | Abc | Ab it would start from the back and
"truncates" to fgeh | geh | ge instead. I hope by doing so I fetch out
the stuff I have not tested as yet, in case I started at some random
spot in the sentence...

Here's a shot from the hip. I half heartedly tossed things together and
[List.Rules:substring]

Post by Demian Smith
[4-9A-Z] val\0 Xal0 '\0 >[3-9A-Y]'\p[4-9A-Z]

The above expands to 1024 rules. It appears to produce all combinations
of min. length 4 except the full original word. I did not test it other
$ ../run/john -pipe -rules:substring -stdout <<< Abcdefgh
efgh
defg
defgh
cdef
cdefg
cdefgh
bcde
bcdef
bcdefg
bcdefgh
magnum

[magnum]

Post by Demian Smith
So, this is my last question and with it I wish to apply for the "Most
stupid question - 2015" award - is it possible, to create a "moving
interval" rule?

Not in the way you describe below, no. Mainly because I believe we can't
handle more than 125 characters of input in one line.

Post by Demian Smith
I am thinking to take my wordlist rule, remove all the line breaks so
that I have a really long stream of chars and then just move the Interval.
Say, my stream would be
SiadqrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
I'd like to have rule (or preoprocessed rule) that walks throught the
file like this
[Siad]qrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
[Siadqrfewmnsohabiwto]arotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqrf]ewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
Si[adqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt

I think this is best done with a perl script or something. Probably very
trivial but I'll pass for now. Anyone?

magnum

[Marek Wrzosek]

Post by magnum

Post by Demian Smith
So, this is my last question and with it I wish to apply for the "Most
stupid question - 2015" award - is it possible, to create a "moving
interval" rule?

Not in the way you describe below, no. Mainly because I believe we can't
handle more than 125 characters of input in one line.

Post by Demian Smith
I am thinking to take my wordlist rule, remove all the line breaks so
that I have a really long stream of chars and then just move the Interval.
Say, my stream would be
SiadqrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
I'd like to have rule (or preoprocessed rule) that walks throught the
file like this
[Siad]qrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
[Siadqrfewmnsohabiwto]arotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqrf]ewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
Si[adqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt

I think this is best done with a perl script or something. Probably very
trivial but I'll pass for now. Anyone?
magnum

Hi

This simple awk one-liner can do the trick:
awk '{for (i = 1; i <= length($1); i++){for
(j=i;j<=length($1);j++){print substr($1, i, j-i+1);}}}'

Feel free to adjust i or j accordingly (e.g. if you don't want passwords
shorter than 4 letters just change j=i to j=i+3).

Best Regards

[Demian Smith]

This is part why I love open source and free software communities,
everybody is so helpful and nice. I am glad to have people like ye and I
am happy whenever I can help other people myself (even though it's not
with JtR though).

No, Marek - if you don't mind, while I understand what awk is, how would
I actually implement the rule you have given me? Would it be |-ed to
John? Or would I pre-process the wordfile? (Totally lost here, sorry...)

Demian

★ On 15/05/17 05:25 p.m. Marek Wrzosek wrote ★
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'

Sinéad O'Connor
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'

Sinéad O'Connor

Post by Marek Wrzosek

Post by magnum

Post by Demian Smith
So, this is my last question and with it I wish to apply for the "Most
stupid question - 2015" award - is it possible, to create a "moving
interval" rule?

Not in the way you describe below, no. Mainly because I believe we can't
handle more than 125 characters of input in one line.

Post by Demian Smith
I am thinking to take my wordlist rule, remove all the line breaks so
that I have a really long stream of chars and then just move the Interval.
Say, my stream would be
SiadqrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
I'd like to have rule (or preoprocessed rule) that walks throught the
file like this
[Siad]qrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
[Siadqrfewmnsohabiwto]arotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqrf]ewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
Si[adqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt

I think this is best done with a perl script or something. Probably very
trivial but I'll pass for now. Anyone?
magnum

Hi
awk '{for (i = 1; i <= length($1); i++){for
(j=i;j<=length($1);j++){print substr($1, i, j-i+1);}}}'
Feel free to adjust i or j accordingly (e.g. if you don't want passwords
shorter than 4 letters just change j=i to j=i+3).
Best Regards

'It's no measure of mental health to be well adjusted
to a profoundly sick society.'

Sinéad O'Connor

[Marek Wrzosek]

Hi, Demian

This awk one-liner will produce very long wordlist, so it depends how
much of free space do you want to waste ;-)
It could be |-ed to john, so no space would be wasted, but I don't know
how fast it will be. If you pre-process the wordlist, it will took a lot
of free space if not compressed, but john will be able to resume after
abort. You could also pre-process the wordlist and gzip it, then use
e.g. zcat to pipe it to john, if you don't like awk. If you decide to
use awk command |-ed to john or decompress wordlist, then I don't know
if john will be able to resume after abort or crash. Maybe magnum will know.

Before you go back to Incremental mode, have you tried Markov mode?

Best Regards

Post by Demian Smith
This is part why I love open source and free software communities,
everybody is so helpful and nice. I am glad to have people like ye and I
am happy whenever I can help other people myself (even though it's not
with JtR though).
No, Marek - if you don't mind, while I understand what awk is, how would
I actually implement the rule you have given me? Would it be |-ed to
John? Or would I pre-process the wordfile? (Totally lost here, sorry...)
Demian
★ On 15/05/17 05:25 p.m. Marek Wrzosek wrote ★
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor

Post by Marek Wrzosek

Post by magnum

Post by Demian Smith
So, this is my last question and with it I wish to apply for the "Most
stupid question - 2015" award - is it possible, to create a "moving
interval" rule?

Not in the way you describe below, no. Mainly because I believe we can't
handle more than 125 characters of input in one line.

Post by Demian Smith
I am thinking to take my wordlist rule, remove all the line breaks so
that I have a really long stream of chars and then just move the Interval.
Say, my stream would be
SiadqrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
I'd like to have rule (or preoprocessed rule) that walks throught the
file like this
[Siad]qrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
[Siadqrfewmnsohabiwto]arotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqrf]ewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
Si[adqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt

I think this is best done with a perl script or something. Probably very
trivial but I'll pass for now. Anyone?
magnum

Hi
awk '{for (i = 1; i <= length($1); i++){for
(j=i;j<=length($1);j++){print substr($1, i, j-i+1);}}}'
Feel free to adjust i or j accordingly (e.g. if you don't want passwords
shorter than 4 letters just change j=i to j=i+3).
Best Regards

'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor

[Demian Smith]

Hi Marek,
thanks for the support - the wordlist is not that big: the source was
102 kb, the resulting file is 105 MB. I am running

john --session=awk --wordlist=/home/demian/RAID/awk/w3
/home/demian/RAID/awk/recover && john --session=awk --rules=truncate
--wordlist=/home/demian/RAID/awk/w3 /home/demian/RAID/awk/recover &&
john --session=awk --rules=substring --wordlist=/home/demian/RAID/awk/w3
/home/demian/RAID/awk/recover

right now to see what happens. Afterwards I will cast the same spell
with --format=tc_ripemd160 and see does it help.

Afterwards I will summon my gaze upon markov, of which I had not heard
previously. But at least all this keeps me occupied and not all hope is
lost.

Thanks you very much,
Demian

★ On 15/05/17 07:00 p.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
Hi, Demian
This awk one-liner will produce very long wordlist, so it depends how
much of free space do you want to waste ;-)
It could be |-ed to john, so no space would be wasted, but I don't know
how fast it will be. If you pre-process the wordlist, it will took a lot
of free space if not compressed, but john will be able to resume after
abort. You could also pre-process the wordlist and gzip it, then use
e.g. zcat to pipe it to john, if you don't like awk. If you decide to
use awk command |-ed to john or decompress wordlist, then I don't know
if john will be able to resume after abort or crash. Maybe magnum will know.
Before you go back to Incremental mode, have you tried Markov mode?
Best Regards

Post by Demian Smith
This is part why I love open source and free software communities,
everybody is so helpful and nice. I am glad to have people like ye and I
am happy whenever I can help other people myself (even though it's not
with JtR though).
No, Marek - if you don't mind, while I understand what awk is, how would
I actually implement the rule you have given me? Would it be |-ed to
John? Or would I pre-process the wordfile? (Totally lost here, sorry...)
Demian
★ On 15/05/17 05:25 p.m. Marek Wrzosek wrote ★
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor

Post by Marek Wrzosek

Post by magnum

Post by Demian Smith
So, this is my last question and with it I wish to apply for the "Most
stupid question - 2015" award - is it possible, to create a "moving
interval" rule?

Not in the way you describe below, no. Mainly because I believe we can't
handle more than 125 characters of input in one line.

Post by Demian Smith
I am thinking to take my wordlist rule, remove all the line breaks so
that I have a really long stream of chars and then just move the Interval.
Say, my stream would be
SiadqrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
I'd like to have rule (or preoprocessed rule) that walks throught the
file like this
[Siad]qrfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
[Siadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
[Siadqrfewmnsohabiwto]arotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadq]rfewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
S[iadqrf]ewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt
(...)
Si[adqr]fewmnsohabiwtoarotwwsbaotmcaaTpthbwuitrubwltetpt

I think this is best done with a perl script or something. Probably very
trivial but I'll pass for now. Anyone?
magnum

Hi
awk '{for (i = 1; i <= length($1); i++){for
(j=i;j<=length($1);j++){print substr($1, i, j-i+1);}}}'
Feel free to adjust i or j accordingly (e.g. if you don't want passwords
shorter than 4 letters just change j=i to j=i+3).
Best Regards

'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor

[Marek Wrzosek]

Hi

I went back to the beginning of this thread. This approach seems right
but if it fails then I don't know if Markov mode is any better than
Incremental.
But if you choose to run Markov mode, you will need to train it with
your wordlist made of pdf. You will also need mask with first capital
letter. If there is a pattern, then Markov model should learn it and
generate passwords very similar to yours. There are two parameters that
are the most important - Markov level and length of passwords. After
creating stats file you'll need to run genmkvpwd command with your stats
file and required password lengths. This command will calculate levels
and number of possible passwords for each level. Depending on how fast
you can crack them and how much time you can spend on cracking, you can
choose right level.
The good news is that using Markov mode you can distribute work among
many computers. And if it fails too, we'll always have Incremental ;-)
But I keep my fingers crossed for Wordlist mode.

Best Regards

Post by Demian Smith
Hi Marek,
thanks for the support - the wordlist is not that big: the source was
102 kb, the resulting file is 105 MB. I am running
john --session=awk --wordlist=/home/demian/RAID/awk/w3
/home/demian/RAID/awk/recover && john --session=awk --rules=truncate
--wordlist=/home/demian/RAID/awk/w3 /home/demian/RAID/awk/recover &&
john --session=awk --rules=substring --wordlist=/home/demian/RAID/awk/w3
/home/demian/RAID/awk/recover
right now to see what happens. Afterwards I will cast the same spell
with --format=tc_ripemd160 and see does it help.
Afterwards I will summon my gaze upon markov, of which I had not heard
previously. But at least all this keeps me occupied and not all hope is
lost.
Thanks you very much,
Demian

[Marek Wrzosek]

Post by Demian Smith
Hi Marek,
thanks for the support - the wordlist is not that big: the source was
102 kb, the resulting file is 105 MB. I am running

Just one more thing. If you are 100% sure that this password is
beginning with capital and is only 5-12 letters long then "walking
interval" filter will be:

awk '{for (i = 1; i <= length($1); i++){for
(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i, j-i+1);}}}'|grep
'^[A-Z]'

The resulting file will be a lot shorter than 105 MB from previous
awk-only command.

Best Regards

[Demian Smith]

Hi Marek,

I tried Markov over night , but it doesn't look really good - I had
trained it on a pwd file generated from the pdf (with keeping only first
letters of each word), but it cretaed mostly candidates like
titttttttttttaief

So, I wanted to get back to the awk version and run it on a similar file
created from the pdfs in the relevant folder, alas, I get
cat all5 | awk '{for (i = 1; i <= length($1);
i++){for(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i,
j-i+1);}}}' | grep'^[A-Z]' > all6

grep^[A-Z]: command not found

I don't know enough about bash programming to sort this one out and
hence would come back to your advise, if you don't mind ...

Thanks,
Demian
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'

Sinéad O'Connor

★ On 15/05/18 04:57 a.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
awk '{for (i = 1; i <= length($1); i++){for
(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i, j-i+1);}}}'|grep
'^[A-Z]'

[Marek Wrzosek]

Hi Demian

There should be space between grep and '^[A-Z]'. The ^[A-Z] regular
expression is for searching lines with a capital as first letter.
I've been thinking about rewriting this awk command to search a capital
letter other than 'I' at the end of string to break the inner loop.
Maybe I'll get rid of grep and change loop type.

Best Regards

Post by Demian Smith
Hi Marek,
I tried Markov over night , but it doesn't look really good - I had
trained it on a pwd file generated from the pdf (with keeping only first
letters of each word), but it cretaed mostly candidates like
titttttttttttaief
So, I wanted to get back to the awk version and run it on a similar file
created from the pdfs in the relevant folder, alas, I get
cat all5 | awk '{for (i = 1; i <= length($1);
i++){for(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i,
j-i+1);}}}' | grep'^[A-Z]' > all6
grep^[A-Z]: command not found
I don't know enough about bash programming to sort this one out and
hence would come back to your advise, if you don't mind ...
Thanks,
Demian
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
★ On 15/05/18 04:57 a.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
awk '{for (i = 1; i <= length($1); i++){for
(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i, j-i+1);}}}'|grep
'^[A-Z]'

[Demian Smith]

Hi Marek,

thanks a mill for the quick reply - slowly this is feeling like I have
taken over the john list :p

I am creating the wordlist right now with your new command, if that
fails, I will run Markov, if that fails as well, I run incremental
forever and a day ...

Post by Marek Wrzosek
I've been thinking about rewriting this awk command to search a capital
letter other than 'I' at the end of string to break the inner loop.
Maybe I'll get rid of grep and change loop type.

I hope all of this will help someone else eventually as well =) and I
can't point out enough how grateful I am for the ongoing help on this list

Best regards and thank you ever so much,
Demian





★ On 15/05/18 07:42 p.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
Hi Demian
There should be space between grep and '^[A-Z]'. The ^[A-Z] regular
expression is for searching lines with a capital as first letter.
I've been thinking about rewriting this awk command to search a capital
letter other than 'I' at the end of string to break the inner loop.
Maybe I'll get rid of grep and change loop type.
Best Regards

Post by Demian Smith
Hi Marek,
I tried Markov over night , but it doesn't look really good - I had
trained it on a pwd file generated from the pdf (with keeping only first
letters of each word), but it cretaed mostly candidates like
titttttttttttaief
So, I wanted to get back to the awk version and run it on a similar file
created from the pdfs in the relevant folder, alas, I get
cat all5 | awk '{for (i = 1; i <= length($1);
i++){for(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i,
j-i+1);}}}' | grep'^[A-Z]' > all6
grep^[A-Z]: command not found
I don't know enough about bash programming to sort this one out and
hence would come back to your advise, if you don't mind ...
Thanks,
Demian
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
★ On 15/05/18 04:57 a.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
awk '{for (i = 1; i <= length($1); i++){for
(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i, j-i+1);}}}'|grep
'^[A-Z]'

[Marek Wrzosek]

Hi

Finally I got it. Two (almost) perfect filters. Almost because they are
made from two awk commands and one has grep in it also. First will print
only once every substring that start with capital letter. First awk
ignores 'I' when searching for next substring, so second awk will repeat
all substrings that start with 'I'.
And this is that first filter:
env LC_COLLATE=C awk '{line=$1;while(match(line,/[A-Z][a-zI]+/)>0){print
substr(line,RSTART,RLENGTH);line=substr(line,RLENGTH+1);}}'|awk
'{line=$1;print
line;while(match(line,"I")>0){if(RSTART>1){line=substr(line,RSTART);print line;}else
line=substr(line,2);}}'

Second filter is almost like that from previous e-mails but it will not
stop after 12 letters. It consists first awk from previous filter but
second awk comes from previous e-mail (with grep).
This is second filter:
env LC_COLLATE=C awk '{line=$1;while(match(line,/[A-Z][a-zI]+/)>0){print
substr(line,RSTART,RLENGTH);line=substr(line,RLENGTH+1);}}'|awk
'{for(i=1;i<=length($1);i++){for(j=i+4;j<=length($1);j++){print
substr($1,i,j-i+1);}}}'|grep '^[A-Z]'

You can use unique command from JtR to get rid of repeated passwords.
I think that passwords from first filter (after unique) can be good as
training sequence for Markov model (but I'm not sure).

Best Regards

Post by Demian Smith
Hi Marek,
thanks a mill for the quick reply - slowly this is feeling like I have
taken over the john list :p
I am creating the wordlist right now with your new command, if that
fails, I will run Markov, if that fails as well, I run incremental
forever and a day ...

Post by Marek Wrzosek
I've been thinking about rewriting this awk command to search a capital
letter other than 'I' at the end of string to break the inner loop.
Maybe I'll get rid of grep and change loop type.

I hope all of this will help someone else eventually as well =) and I
can't point out enough how grateful I am for the ongoing help on this list
Best regards and thank you ever so much,
Demian
★ On 15/05/18 07:42 p.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
Hi Demian
There should be space between grep and '^[A-Z]'. The ^[A-Z] regular
expression is for searching lines with a capital as first letter.
I've been thinking about rewriting this awk command to search a capital
letter other than 'I' at the end of string to break the inner loop.
Maybe I'll get rid of grep and change loop type.
Best Regards

Post by Demian Smith
Hi Marek,
I tried Markov over night , but it doesn't look really good - I had
trained it on a pwd file generated from the pdf (with keeping only first
letters of each word), but it cretaed mostly candidates like
titttttttttttaief
So, I wanted to get back to the awk version and run it on a similar file
created from the pdfs in the relevant folder, alas, I get
cat all5 | awk '{for (i = 1; i <= length($1);
i++){for(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i,
j-i+1);}}}' | grep'^[A-Z]' > all6
grep^[A-Z]: command not found
I don't know enough about bash programming to sort this one out and
hence would come back to your advise, if you don't mind ...
Thanks,
Demian
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
★ On 15/05/18 04:57 a.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
awk '{for (i = 1; i <= length($1); i++){for
(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i, j-i+1);}}}'|grep
'^[A-Z]'

[Demian Smith]

Thanks a million, Marek, that’s reaaaaly great work.

I run it on my "allpdf" file as well as on the relevant pdf#s of the
books. I'm pretty confident that by now it should have been cracked,
alas, it isn't.

I right now run the two filters on the first txt file I create from the
suspect pdf and will then go back to incremental, as the Markov mode -
in my case - does not appear to be producing useful candidates.

Thanks again for all the effort, I'm pretty sure this is a layer 8 issue
right now :s

Demian

★ On 15/05/18 10:17 p.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
Hi
Finally I got it. Two (almost) perfect filters. Almost because they are
made from two awk commands and one has grep in it also. First will print
only once every substring that start with capital letter. First awk
ignores 'I' when searching for next substring, so second awk will repeat
all substrings that start with 'I'.
env LC_COLLATE=C awk '{line=$1;while(match(line,/[A-Z][a-zI]+/)>0){print
substr(line,RSTART,RLENGTH);line=substr(line,RLENGTH+1);}}'|awk
'{line=$1;print
line;while(match(line,"I")>0){if(RSTART>1){line=substr(line,RSTART);print line;}else
line=substr(line,2);}}'
Second filter is almost like that from previous e-mails but it will not
stop after 12 letters. It consists first awk from previous filter but
second awk comes from previous e-mail (with grep).
env LC_COLLATE=C awk '{line=$1;while(match(line,/[A-Z][a-zI]+/)>0){print
substr(line,RSTART,RLENGTH);line=substr(line,RLENGTH+1);}}'|awk
'{for(i=1;i<=length($1);i++){for(j=i+4;j<=length($1);j++){print
substr($1,i,j-i+1);}}}'|grep '^[A-Z]'
You can use unique command from JtR to get rid of repeated passwords.
I think that passwords from first filter (after unique) can be good as
training sequence for Markov model (but I'm not sure).
Best Regards

Post by Demian Smith
Hi Marek,
thanks a mill for the quick reply - slowly this is feeling like I have
taken over the john list :p
I am creating the wordlist right now with your new command, if that
fails, I will run Markov, if that fails as well, I run incremental
forever and a day ...

Post by Marek Wrzosek
I've been thinking about rewriting this awk command to search a capital
letter other than 'I' at the end of string to break the inner loop.
Maybe I'll get rid of grep and change loop type.

I hope all of this will help someone else eventually as well =) and I
can't point out enough how grateful I am for the ongoing help on this list
Best regards and thank you ever so much,
Demian
★ On 15/05/18 07:42 p.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
Hi Demian
There should be space between grep and '^[A-Z]'. The ^[A-Z] regular
expression is for searching lines with a capital as first letter.
I've been thinking about rewriting this awk command to search a capital
letter other than 'I' at the end of string to break the inner loop.
Maybe I'll get rid of grep and change loop type.
Best Regards

Post by Demian Smith
Hi Marek,
I tried Markov over night , but it doesn't look really good - I had
trained it on a pwd file generated from the pdf (with keeping only first
letters of each word), but it cretaed mostly candidates like
titttttttttttaief
So, I wanted to get back to the awk version and run it on a similar file
created from the pdfs in the relevant folder, alas, I get
cat all5 | awk '{for (i = 1; i <= length($1);
i++){for(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i,
j-i+1);}}}' | grep'^[A-Z]' > all6
grep^[A-Z]: command not found
I don't know enough about bash programming to sort this one out and
hence would come back to your advise, if you don't mind ...
Thanks,
Demian
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
★ On 15/05/18 04:57 a.m. Marek Wrzosek wrote ★

Post by Marek Wrzosek
awk '{for (i = 1; i <= length($1); i++){for
(j=i+4;j<i+12&&j<=length($1);j++){print substr($1, i, j-i+1);}}}'|grep
'^[A-Z]'

[magnum]

Post by Demian Smith
I right now run the two filters on the first txt file I create from the
suspect pdf and will then go back to incremental, as the Markov mode -
in my case - does not appear to be producing useful candidates.
Thanks again for all the effort, I'm pretty sure this is a layer 8 issue
right now :s

Maybe we should revert to verifying your truerypt_volume2john
invocation/results.

Please recap what you had, what you did and what you got. Were you
feeding truecrypt_volume2john a file or a device special node? Was there
any output on stderr? How does your "hash" file look? I still wonder why
you got two "hashes".

magnum

[Demian Smith]

Not all hope is lost, so?

So, what I did was:

attach the external device to usb and verify it's "path" via lsblk
and/or truecryp. This led to
sdb 8:16 0 465.7G 0 disk
└─sdb1 8:17 0 465.7G 0 part

I than do:
[***@anonymous:~/.bin/JtR/run]$ ./truecrypt_volume2john /dev/sdb1 >
~/hash

which results in the attached hash file.

I had tried the same on a usb key, as well running truecrypt2john versus
the partition on sdb1, which then had been "cracked"...

If I create a hashfile on /dev/sdb instead, I get

john --session=wl --wordlist=/home/wpd_for_mark_second.txt ~/no_partition
ASCII -> ASCII -> ASCII
Warning: detected hash type "tc_aes_xts", but the string is also
recognized as "tc_ripemd160"
Use the "--format=tc_ripemd160" option to force loading these as that
type instead
Loaded 6 password hashes with 6 different salts (tc_aes_xts, TrueCrypt
AES256_XTS [SHA512 128/128 SSE4.1 2x /RIPEMD160/WHIRLPOOL])
Loaded hashes with cost 1 (hash algorithm [1:SHA512 2:RIPEMD160
3:Whirlpool]) varying from 1 to 3
Will run 4 OpenMP threads

If I force ripemd w/ --format=tc_ripemd160:
initUnicode(UNICODE,ASCII/ASCII)


ASCII->ASCII->ASCII


Loaded 2 password hashes with 2 different salts (tc_ripemd160, TrueCrypt
AES256_XTS [RIPEMD160 32/64])

Will run 4 OpenMP threads

while the hashfile itself looks different ...

i did look into the doc folder but could not spot anything related to
truecrypt, I hope I did not just miss it...

Also, I hope I just made a mistake somewhere on the lines of generating
the hashes, maybe ...

Thanks for keeping my hopes up,
D
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'

Sinéad O'Connor

★ On 15/05/19 09:00 p.m. Magnum wrote ★

Post by magnum

Post by Demian Smith
I right now run the two filters on the first txt file I create from the
suspect pdf and will then go back to incremental, as the Markov mode -
in my case - does not appear to be producing useful candidates.
Thanks again for all the effort, I'm pretty sure this is a layer 8 issue
right now :s

Maybe we should revert to verifying your truerypt_volume2john
invocation/results.
Please recap what you had, what you did and what you got. Were you
feeding truecrypt_volume2john a file or a device special node? Was there
any output on stderr? How does your "hash" file look? I still wonder why
you got two "hashes".
magnum

[magnum]

The hash_nopart one from sdb has entries with mostly nulls so is not
likely correct. I just made some quick tests and reviewed the code.
Unfortunately, truecrypt_volume2john doesn't recognize any signature or
magic, so you can feed it ANY data and as long as it's at least 512
bytes it will happily produce six different hashes: One for each of the
three algorithms, plus another set of three for possible hidden volumes.
I believe there's no way to make it any better - truecrypt is designed
to be plausably deniable. Of the six ones always produced, I believe
only one is ever correct (if any) and often you won't know which.

I tried running truecrypt_volume2john on some actual truecrypt container
(file) I had lying around. Obviously it "found" six hashes in it too.
When trying with the correct password, the correct one was cracked
(apparently it was a RIPEMD_160 one) while the other ones are bogus
random data that can't ever be cracked.

So, if last weeks attempts were made using the file from sbd1, and
without using the --format option no narrow down to a particular algo, I
think you did correct. If you did narrow it down to eg.
--format=tc_ripemd160, I hope that's because you KNOW that is the algo used.

magnum

Post by Demian Smith
Not all hope is lost, so?
attach the external device to usb and verify it's "path" via lsblk
and/or truecryp. This led to
sdb 8:16 0 465.7G 0 disk
└─sdb1 8:17 0 465.7G 0 part
~/hash
which results in the attached hash file.
I had tried the same on a usb key, as well running truecrypt2john versus
the partition on sdb1, which then had been "cracked"...
If I create a hashfile on /dev/sdb instead, I get
john --session=wl --wordlist=/home/wpd_for_mark_second.txt ~/no_partition
ASCII -> ASCII -> ASCII
Warning: detected hash type "tc_aes_xts", but the string is also
recognized as "tc_ripemd160"
Use the "--format=tc_ripemd160" option to force loading these as that
type instead
Loaded 6 password hashes with 6 different salts (tc_aes_xts, TrueCrypt
AES256_XTS [SHA512 128/128 SSE4.1 2x /RIPEMD160/WHIRLPOOL])
Loaded hashes with cost 1 (hash algorithm [1:SHA512 2:RIPEMD160
3:Whirlpool]) varying from 1 to 3
Will run 4 OpenMP threads
initUnicode(UNICODE,ASCII/ASCII)
ASCII->ASCII->ASCII
Loaded 2 password hashes with 2 different salts (tc_ripemd160, TrueCrypt
AES256_XTS [RIPEMD160 32/64])
Will run 4 OpenMP threads
while the hashfile itself looks different ...
i did look into the doc folder but could not spot anything related to
truecrypt, I hope I did not just miss it...
Also, I hope I just made a mistake somewhere on the lines of generating
the hashes, maybe ...
Thanks for keeping my hopes up,
D
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
★ On 15/05/19 09:00 p.m. Magnum wrote ★

Post by magnum

Post by Demian Smith
I right now run the two filters on the first txt file I create from the
suspect pdf and will then go back to incremental, as the Markov mode -
in my case - does not appear to be producing useful candidates.
Thanks again for all the effort, I'm pretty sure this is a layer 8 issue
right now :s

Maybe we should revert to verifying your truerypt_volume2john
invocation/results.
Please recap what you had, what you did and what you got. Were you
feeding truecrypt_volume2john a file or a device special node? Was there
any output on stderr? How does your "hash" file look? I still wonder why
you got two "hashes".
magnum

[Demian Smith]

Hi magnum,

thank you for looking into it for me - the plausible deniability (?) is
a nice feature and I believe it's one that it is good to have.

In the last weeks I did run with the --format option, as I think I would
not have changed the defaults. Which - on my linux machine - are ripemd
and aes. I hope they would have been the same on a win machine, but
can't verify, as I don't have an old truecrypt copy lying around.

So, I have removed the hidden-container lines from the file (I know
there's no hidden volume) and am now back to running the wordfile onto
it and then incremental... Will keep my finger's crossed though =)

Cheers,
Demian

★ On 15/05/20 01:12 a.m. Magnum wrote ★

Post by magnum
The hash_nopart one from sdb has entries with mostly nulls so is not
likely correct. I just made some quick tests and reviewed the code.
Unfortunately, truecrypt_volume2john doesn't recognize any signature or
magic, so you can feed it ANY data and as long as it's at least 512
bytes it will happily produce six different hashes: One for each of the
three algorithms, plus another set of three for possible hidden volumes.
I believe there's no way to make it any better - truecrypt is designed
to be plausably deniable. Of the six ones always produced, I believe
only one is ever correct (if any) and often you won't know which.
I tried running truecrypt_volume2john on some actual truecrypt container
(file) I had lying around. Obviously it "found" six hashes in it too.
When trying with the correct password, the correct one was cracked
(apparently it was a RIPEMD_160 one) while the other ones are bogus
random data that can't ever be cracked.
So, if last weeks attempts were made using the file from sbd1, and
without using the --format option no narrow down to a particular algo, I
think you did correct. If you did narrow it down to eg.
--format=tc_ripemd160, I hope that's because you KNOW that is the algo used.
magnum

Post by Demian Smith
Not all hope is lost, so?
attach the external device to usb and verify it's "path" via lsblk
and/or truecryp. This led to
sdb 8:16 0 465.7G 0 disk
└─sdb1 8:17 0 465.7G 0 part
~/hash
which results in the attached hash file.
I had tried the same on a usb key, as well running truecrypt2john versus
the partition on sdb1, which then had been "cracked"...
If I create a hashfile on /dev/sdb instead, I get
john --session=wl --wordlist=/home/wpd_for_mark_second.txt ~/no_partition
ASCII -> ASCII -> ASCII
Warning: detected hash type "tc_aes_xts", but the string is also
recognized as "tc_ripemd160"
Use the "--format=tc_ripemd160" option to force loading these as that
type instead
Loaded 6 password hashes with 6 different salts (tc_aes_xts, TrueCrypt
AES256_XTS [SHA512 128/128 SSE4.1 2x /RIPEMD160/WHIRLPOOL])
Loaded hashes with cost 1 (hash algorithm [1:SHA512 2:RIPEMD160
3:Whirlpool]) varying from 1 to 3
Will run 4 OpenMP threads
initUnicode(UNICODE,ASCII/ASCII)
ASCII->ASCII->ASCII
Loaded 2 password hashes with 2 different salts (tc_ripemd160, TrueCrypt
AES256_XTS [RIPEMD160 32/64])
Will run 4 OpenMP threads
while the hashfile itself looks different ...
i did look into the doc folder but could not spot anything related to
truecrypt, I hope I did not just miss it...
Also, I hope I just made a mistake somewhere on the lines of generating
the hashes, maybe ...
Thanks for keeping my hopes up,
D
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
★ On 15/05/19 09:00 p.m. Magnum wrote ★

Post by magnum

Post by Demian Smith
I right now run the two filters on the first txt file I create from the
suspect pdf and will then go back to incremental, as the Markov mode -
in my case - does not appear to be producing useful candidates.
Thanks again for all the effort, I'm pretty sure this is a layer 8 issue
right now :s

Maybe we should revert to verifying your truerypt_volume2john
invocation/results.
Please recap what you had, what you did and what you got. Were you
feeding truecrypt_volume2john a file or a device special node? Was there
any output on stderr? How does your "hash" file look? I still wonder why
you got two "hashes".
magnum

[Marek Wrzosek]

Hi Demian

What version of truecrypt did you use on windows?

Post by Demian Smith
Hi magnum,
thank you for looking into it for me - the plausible deniability (?) is
a nice feature and I believe it's one that it is good to have.
In the last weeks I did run with the --format option, as I think I would
not have changed the defaults. Which - on my linux machine - are ripemd
and aes. I hope they would have been the same on a win machine, but
can't verify, as I don't have an old truecrypt copy lying around.
So, I have removed the hidden-container lines from the file (I know
there's no hidden volume) and am now back to running the wordfile onto
it and then incremental... Will keep my finger's crossed though =)
Cheers,
Demian
★ On 15/05/20 01:12 a.m. Magnum wrote ★

Post by magnum
The hash_nopart one from sdb has entries with mostly nulls so is not
likely correct. I just made some quick tests and reviewed the code.
Unfortunately, truecrypt_volume2john doesn't recognize any signature or
magic, so you can feed it ANY data and as long as it's at least 512
bytes it will happily produce six different hashes: One for each of the
three algorithms, plus another set of three for possible hidden volumes.
I believe there's no way to make it any better - truecrypt is designed
to be plausably deniable. Of the six ones always produced, I believe
only one is ever correct (if any) and often you won't know which.
I tried running truecrypt_volume2john on some actual truecrypt container
(file) I had lying around. Obviously it "found" six hashes in it too.
When trying with the correct password, the correct one was cracked
(apparently it was a RIPEMD_160 one) while the other ones are bogus
random data that can't ever be cracked.
So, if last weeks attempts were made using the file from sbd1, and
without using the --format option no narrow down to a particular algo, I
think you did correct. If you did narrow it down to eg.
--format=tc_ripemd160, I hope that's because you KNOW that is the algo used.
magnum

Post by Demian Smith
Not all hope is lost, so?
attach the external device to usb and verify it's "path" via lsblk
and/or truecryp. This led to
sdb 8:16 0 465.7G 0 disk
└─sdb1 8:17 0 465.7G 0 part
~/hash
which results in the attached hash file.
I had tried the same on a usb key, as well running truecrypt2john versus
the partition on sdb1, which then had been "cracked"...
If I create a hashfile on /dev/sdb instead, I get
john --session=wl --wordlist=/home/wpd_for_mark_second.txt ~/no_partition
ASCII -> ASCII -> ASCII
Warning: detected hash type "tc_aes_xts", but the string is also
recognized as "tc_ripemd160"
Use the "--format=tc_ripemd160" option to force loading these as that
type instead
Loaded 6 password hashes with 6 different salts (tc_aes_xts, TrueCrypt
AES256_XTS [SHA512 128/128 SSE4.1 2x /RIPEMD160/WHIRLPOOL])
Loaded hashes with cost 1 (hash algorithm [1:SHA512 2:RIPEMD160
3:Whirlpool]) varying from 1 to 3
Will run 4 OpenMP threads
initUnicode(UNICODE,ASCII/ASCII)
ASCII->ASCII->ASCII
Loaded 2 password hashes with 2 different salts (tc_ripemd160, TrueCrypt
AES256_XTS [RIPEMD160 32/64])
Will run 4 OpenMP threads
while the hashfile itself looks different ...
i did look into the doc folder but could not spot anything related to
truecrypt, I hope I did not just miss it...
Also, I hope I just made a mistake somewhere on the lines of generating
the hashes, maybe ...
Thanks for keeping my hopes up,
D
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
★ On 15/05/19 09:00 p.m. Magnum wrote ★

Post by magnum

Post by Demian Smith
I right now run the two filters on the first txt file I create from the
suspect pdf and will then go back to incremental, as the Markov mode -
in my case - does not appear to be producing useful candidates.
Thanks again for all the effort, I'm pretty sure this is a layer 8 issue
right now :s

Maybe we should revert to verifying your truerypt_volume2john
invocation/results.
Please recap what you had, what you did and what you got. Were you
feeding truecrypt_volume2john a file or a device special node? Was there
any output on stderr? How does your "hash" file look? I still wonder why
you got two "hashes".
magnum

[Demian Smith]

Hi Marek,
I believe it would have been 6.2, which had been the latest update, but
it's not available for download from the former official website anymore ...

6.2 is as well the one running on my ubuntu box, so I have they're
comparable ...
--
★ On 15/05/20 05:59 p.m. Marek Wrzosek wrote ★
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'

Sinéad O'Connor

Post by Marek Wrzosek
Hi Demian
What version of truecrypt did you use on windows?

Post by Demian Smith
Hi magnum,
thank you for looking into it for me - the plausible deniability (?) is
a nice feature and I believe it's one that it is good to have.
In the last weeks I did run with the --format option, as I think I would
not have changed the defaults. Which - on my linux machine - are ripemd
and aes. I hope they would have been the same on a win machine, but
can't verify, as I don't have an old truecrypt copy lying around.
So, I have removed the hidden-container lines from the file (I know
there's no hidden volume) and am now back to running the wordfile onto
it and then incremental... Will keep my finger's crossed though =)
Cheers,
Demian
★ On 15/05/20 01:12 a.m. Magnum wrote ★

Post by magnum
The hash_nopart one from sdb has entries with mostly nulls so is not
likely correct. I just made some quick tests and reviewed the code.
Unfortunately, truecrypt_volume2john doesn't recognize any signature or
magic, so you can feed it ANY data and as long as it's at least 512
bytes it will happily produce six different hashes: One for each of the
three algorithms, plus another set of three for possible hidden volumes.
I believe there's no way to make it any better - truecrypt is designed
to be plausably deniable. Of the six ones always produced, I believe
only one is ever correct (if any) and often you won't know which.
I tried running truecrypt_volume2john on some actual truecrypt container
(file) I had lying around. Obviously it "found" six hashes in it too.
When trying with the correct password, the correct one was cracked
(apparently it was a RIPEMD_160 one) while the other ones are bogus
random data that can't ever be cracked.
So, if last weeks attempts were made using the file from sbd1, and
without using the --format option no narrow down to a particular algo, I
think you did correct. If you did narrow it down to eg.
--format=tc_ripemd160, I hope that's because you KNOW that is the algo used.
magnum

Post by Demian Smith
Not all hope is lost, so?
attach the external device to usb and verify it's "path" via lsblk
and/or truecryp. This led to
sdb 8:16 0 465.7G 0 disk
└─sdb1 8:17 0 465.7G 0 part
~/hash
which results in the attached hash file.
I had tried the same on a usb key, as well running truecrypt2john versus
the partition on sdb1, which then had been "cracked"...
If I create a hashfile on /dev/sdb instead, I get
john --session=wl --wordlist=/home/wpd_for_mark_second.txt ~/no_partition
ASCII -> ASCII -> ASCII
Warning: detected hash type "tc_aes_xts", but the string is also
recognized as "tc_ripemd160"
Use the "--format=tc_ripemd160" option to force loading these as that
type instead
Loaded 6 password hashes with 6 different salts (tc_aes_xts, TrueCrypt
AES256_XTS [SHA512 128/128 SSE4.1 2x /RIPEMD160/WHIRLPOOL])
Loaded hashes with cost 1 (hash algorithm [1:SHA512 2:RIPEMD160
3:Whirlpool]) varying from 1 to 3
Will run 4 OpenMP threads
initUnicode(UNICODE,ASCII/ASCII)
ASCII->ASCII->ASCII
Loaded 2 password hashes with 2 different salts (tc_ripemd160, TrueCrypt
AES256_XTS [RIPEMD160 32/64])
Will run 4 OpenMP threads
while the hashfile itself looks different ...
i did look into the doc folder but could not spot anything related to
truecrypt, I hope I did not just miss it...
Also, I hope I just made a mistake somewhere on the lines of generating
the hashes, maybe ...
Thanks for keeping my hopes up,
D
--
'It's no measure of mental health to be well adjusted
to a profoundly sick society.'
Sinéad O'Connor
★ On 15/05/19 09:00 p.m. Magnum wrote ★

Post by magnum

Post by Demian Smith
I right now run the two filters on the first txt file I create from the
suspect pdf and will then go back to incremental, as the Markov mode -
in my case - does not appear to be producing useful candidates.
Thanks again for all the effort, I'm pretty sure this is a layer 8 issue
right now :s

Maybe we should revert to verifying your truerypt_volume2john
invocation/results.
Please recap what you had, what you did and what you got. Were you
feeding truecrypt_volume2john a file or a device special node? Was there
any output on stderr? How does your "hash" file look? I still wonder why
you got two "hashes".
magnum

[Demian Smith]

Hi guys,
I thought you might be interested in me getting on - the pw is still not
recovered and I start to wonder did I really use ripemd160 ^^

My current status is:

john --restore=raid_ripemdd
initUnicode(UNICODE, ASCII/ASCII)
ASCII -> ASCII -> ASCII
Loaded 1 password hash (tc_ripemd160, TrueCrypt AES256_XTS [RIPEMD160
32/64])
Will run 4 OpenMP threads
0g 1019464304p 46:15:34:02 0g/s 252.9p/s 252.9c/s 252.9C/s
Hihfaysbf..Hihfaysba

I still just let it run whenever the laptop is on and over night, but it
might be futile altogether ...

Best regards (and thanks again for all the great help and tips),
Demian